TimeMoEOptions<T>

Options & Configuration · AiDotNet.Models.Options

Configuration options for Time-MoE (Billion-Scale Time Series Foundation Models with Mixture of Experts).

Inheritance: ModelOptions → RegressionOptions<T> → TimeSeriesRegressionOptions<T> → TimeMoEOptions<T>

For Beginners

Time-MoE achieves massive scale efficiently:

Mixture of Experts: Instead of using one large feed-forward network, MoE uses multiple smaller “expert” networks and a router that selects which experts to use for each input. This means the model has many parameters but only uses a fraction for each prediction.

Model Sizes:

  • 50M parameters (all active)
  • 200M total / ~50M active per token
  • 2.4B total / ~300M active per token (largest)

How It Works

Time-MoE is the first billion-scale time series foundation model, using sparse Mixture of Experts (MoE) for efficient scaling up to 2.4B parameters. It uses a decoder-only transformer with MoE feed-forward layers.

Reference: Shi et al., “Time-MoE: Billion-Scale Time Series Foundation Models with Mixture of Experts”, ICLR 2025. https://openreview.net/forum?id=e1wDDFmlVu

Constructors

Constructor Summary
TimeMoEOptions() Initializes a new instance with default values.
TimeMoEOptions(TimeMoEOptions<T> other) Initializes a new instance by copying from another instance.

Properties

Property Summary
FoundationModelSize ModelSize { get; set; } Gets or sets the model size variant.
double DropoutRate { get; set; } Gets or sets the dropout rate.
double RouterAuxLossWeight { get; set; } Gets or sets the auxiliary loss weight for the router load balancing.
int ContextLength { get; set; } Gets or sets the context length.
int ForecastHorizon { get; set; } Gets or sets the forecast horizon.
int HiddenDimension { get; set; } Gets or sets the hidden dimension.
int IntermediateSize { get; set; } Gets or sets the intermediate size per expert.
int NumActiveExperts { get; set; } Gets or sets the number of active experts per token.
int NumExperts { get; set; } Gets or sets the total number of experts in each MoE layer.
int NumHeads { get; set; } Gets or sets the number of attention heads.
int NumLayers { get; set; } Gets or sets the number of transformer layers.
int PatchLength { get; set; } Gets or sets the patch length.