TimeMoE<T>

Models & Types · AiDotNet.Finance.Forecasting.Foundation

Time-MoE — Billion-Scale Time Series Foundation Models with Mixture of Experts.

Inheritance: NeuralNetworkBase<T> → FinancialModelBase<T> → ForecastingModelBase<T> → TimeSeriesFoundationModelBase<T> → TimeMoE<T>

For Beginners

Time-MoE is the first time series model to reach billions of parameters by using a Mixture of Experts approach. Instead of one giant network processing every input, it has many specialized “expert” sub-networks and a router that picks the best 2-3 experts for each data point. This means only a fraction of the parameters are active at any time, making it efficient despite its massive total size.

How It Works

Time-MoE is the first billion-scale time series foundation model, using sparse Mixture of Experts for efficient scaling up to 2.4B total parameters (~300M active per token). It uses a decoder-only transformer where each feed-forward layer is replaced by an MoE layer with a learned router.

Reference: Shi et al., “Time-MoE: Billion-Scale Time Series Foundation Models with Mixture of Experts”, ICLR 2025. https://openreview.net/forum?id=e1wDDFmlVu

Example

// Create a Time-MoE billion-scale foundation model with Mixture of Experts
// 2.4B total parameters with only ~300M active per token via sparse expert routing
var architecture = new NeuralNetworkArchitecture<double>(
    inputType: InputType.OneDimensional,
    taskType: NeuralNetworkTaskType.Regression,
    inputHeight: 512, inputWidth: 1, inputDepth: 1, outputSize: 24);
            
// Training mode with MoE layers and learned expert routing
var model = new TimeMoE<double>(architecture);
            
// ONNX inference mode with pre-trained model
var onnxModel = new TimeMoE<double>(architecture, "time_moe.onnx");

Constructors

Constructor Summary
TimeMoE(NeuralNetworkArchitecture<T> architecture, TimeMoEOptions<T> options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer, ILossFunction<T> lossFunction) Creates a Time-MoE model in native mode.
TimeMoE(NeuralNetworkArchitecture<T> architecture, string onnxModelPath, TimeMoEOptions<T> options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer, ILossFunction<T> lossFunction) Creates a Time-MoE model using a pretrained ONNX model.

Properties

Property Summary
FoundationModelSize ModelSize { get; }
bool IsChannelIndependent { get; }
bool SupportsTraining { get; }
bool UseNativeMode { get; }
int MaxContextLength { get; }
int MaxPredictionHorizon { get; }
int NumFeatures { get; }
int PatchSize { get; }
int PredictionHorizon { get; }
int SequenceLength { get; }
int Stride { get; }

Methods

Method Summary
Dictionary<string, T> Evaluate(Tensor<T> predictions, Tensor<T> actuals)
Dictionary<string, T> GetFinancialMetrics()
Dictionary<string, Tensor<T>> GetNamedLayerActivations(Tensor<T> input)
ModelMetadata<T> GetModelMetadata()
Tensor<T> ApplyInstanceNormalization(Tensor<T> input)
Tensor<T> AutoregressiveForecast(Tensor<T> input, int steps)