TimeMoE<T>
Models & Types · AiDotNet.Finance.Forecasting.Foundation
Time-MoE — Billion-Scale Time Series Foundation Models with Mixture of Experts.
Inheritance: NeuralNetworkBase<T> → FinancialModelBase<T> → ForecastingModelBase<T> → TimeSeriesFoundationModelBase<T> → TimeMoE<T>
For Beginners
Time-MoE is the first time series model to reach billions of parameters by using a Mixture of Experts approach. Instead of one giant network processing every input, it has many specialized “expert” sub-networks and a router that picks the best 2-3 experts for each data point. This means only a fraction of the parameters are active at any time, making it efficient despite its massive total size.
How It Works
Time-MoE is the first billion-scale time series foundation model, using sparse Mixture of Experts for efficient scaling up to 2.4B total parameters (~300M active per token). It uses a decoder-only transformer where each feed-forward layer is replaced by an MoE layer with a learned router.
Reference: Shi et al., “Time-MoE: Billion-Scale Time Series Foundation Models
with Mixture of Experts”, ICLR 2025. https://openreview.net/forum?id=e1wDDFmlVu
Example
// Create a Time-MoE billion-scale foundation model with Mixture of Experts
// 2.4B total parameters with only ~300M active per token via sparse expert routing
var architecture = new NeuralNetworkArchitecture<double>(
inputType: InputType.OneDimensional,
taskType: NeuralNetworkTaskType.Regression,
inputHeight: 512, inputWidth: 1, inputDepth: 1, outputSize: 24);
// Training mode with MoE layers and learned expert routing
var model = new TimeMoE<double>(architecture);
// ONNX inference mode with pre-trained model
var onnxModel = new TimeMoE<double>(architecture, "time_moe.onnx");
Constructors
| Constructor | Summary |
|---|---|
TimeMoE(NeuralNetworkArchitecture<T> architecture, TimeMoEOptions<T> options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer, ILossFunction<T> lossFunction) |
Creates a Time-MoE model in native mode. |
TimeMoE(NeuralNetworkArchitecture<T> architecture, string onnxModelPath, TimeMoEOptions<T> options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer, ILossFunction<T> lossFunction) |
Creates a Time-MoE model using a pretrained ONNX model. |
Properties
| Property | Summary |
|---|---|
FoundationModelSize ModelSize { get; } |
|
bool IsChannelIndependent { get; } |
|
bool SupportsTraining { get; } |
|
bool UseNativeMode { get; } |
|
int MaxContextLength { get; } |
|
int MaxPredictionHorizon { get; } |
|
int NumFeatures { get; } |
|
int PatchSize { get; } |
|
int PredictionHorizon { get; } |
|
int SequenceLength { get; } |
|
int Stride { get; } |
Methods
| Method | Summary |
|---|---|
Dictionary<string, T> Evaluate(Tensor<T> predictions, Tensor<T> actuals) |
|
Dictionary<string, T> GetFinancialMetrics() |
|
Dictionary<string, Tensor<T>> GetNamedLayerActivations(Tensor<T> input) |
|
ModelMetadata<T> GetModelMetadata() |
|
Tensor<T> ApplyInstanceNormalization(Tensor<T> input) |
|
Tensor<T> AutoregressiveForecast(Tensor<T> input, int steps) |