TimeMoEOptions<T>
Options & Configuration · AiDotNet.Models.Options
Configuration options for Time-MoE (Billion-Scale Time Series Foundation Models with Mixture of Experts).
Inheritance: ModelOptions → RegressionOptions<T> → TimeSeriesRegressionOptions<T> → TimeMoEOptions<T>
For Beginners
Time-MoE achieves massive scale efficiently:
Mixture of Experts: Instead of using one large feed-forward network, MoE uses multiple smaller “expert” networks and a router that selects which experts to use for each input. This means the model has many parameters but only uses a fraction for each prediction.
Model Sizes:
- 50M parameters (all active)
- 200M total / ~50M active per token
- 2.4B total / ~300M active per token (largest)
How It Works
Time-MoE is the first billion-scale time series foundation model, using sparse Mixture of Experts (MoE) for efficient scaling up to 2.4B parameters. It uses a decoder-only transformer with MoE feed-forward layers.
Reference: Shi et al., “Time-MoE: Billion-Scale Time Series Foundation Models
with Mixture of Experts”, ICLR 2025. https://openreview.net/forum?id=e1wDDFmlVu
Constructors
| Constructor | Summary |
|---|---|
TimeMoEOptions() |
Initializes a new instance with default values. |
TimeMoEOptions(TimeMoEOptions<T> other) |
Initializes a new instance by copying from another instance. |
Properties
| Property | Summary |
|---|---|
FoundationModelSize ModelSize { get; set; } |
Gets or sets the model size variant. |
double DropoutRate { get; set; } |
Gets or sets the dropout rate. |
double RouterAuxLossWeight { get; set; } |
Gets or sets the auxiliary loss weight for the router load balancing. |
int ContextLength { get; set; } |
Gets or sets the context length. |
int ForecastHorizon { get; set; } |
Gets or sets the forecast horizon. |
int HiddenDimension { get; set; } |
Gets or sets the hidden dimension. |
int IntermediateSize { get; set; } |
Gets or sets the intermediate size per expert. |
int NumActiveExperts { get; set; } |
Gets or sets the number of active experts per token. |
int NumExperts { get; set; } |
Gets or sets the total number of experts in each MoE layer. |
int NumHeads { get; set; } |
Gets or sets the number of attention heads. |
int NumLayers { get; set; } |
Gets or sets the number of transformer layers. |
int PatchLength { get; set; } |
Gets or sets the patch length. |