Mamba2Options<T>
Options & Configuration · AiDotNet.Models.Options
Configuration options for Mamba-2 (Structured State Space Duality) forecasting.
Inheritance: ModelOptions → RegressionOptions<T> → TimeSeriesRegressionOptions<T> → Mamba2Options<T>
For Beginners
Mamba-2 is an evolution of the Mamba architecture:
Key Improvements over Mamba-1:
- SSD Algorithm: Uses matrix multiply instead of associative scan — much faster on GPUs
- Multi-head Structure: Like multi-head attention, enabling better capacity per parameter
- Chunk-wise Processing: Processes sequences in chunks for better hardware utilization
- 2-8x Faster Training: Due to better hardware mapping
Architecture:
- Input projection to expanded dimension
- Multi-head structured state space blocks
- Chunk-wise parallel processing
- Output projection back to model dimension
How It Works
Mamba-2 improves upon Mamba by discovering the connection between selective state space models and structured masked attention (State Space Duality). This enables a more efficient SSD algorithm using matrix multiplications rather than associative scans, achieving 2-8x faster training.
Reference: Dao and Gu, “Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Duality”, 2024.
Constructors
| Constructor | Summary |
|---|---|
Mamba2Options() |
Initializes a new instance with default values. |
Mamba2Options(Mamba2Options<T> other) |
Initializes a new instance by copying from another instance. |
Properties
| Property | Summary |
|---|---|
double DropoutRate { get; set; } |
Gets or sets the dropout rate for regularization. |
int ChunkSize { get; set; } |
Gets or sets the chunk size for SSD computation. |
int ContextLength { get; set; } |
Gets or sets the context length (input sequence length). |
int ConvKernelSize { get; set; } |
Gets or sets the convolution kernel size. |
int ExpandFactor { get; set; } |
Gets or sets the expansion factor for the inner dimension. |
int ForecastHorizon { get; set; } |
Gets or sets the forecast horizon (prediction length). |
int ModelDimension { get; set; } |
Gets or sets the model dimension (d_model). |
int NumHeads { get; set; } |
Gets or sets the number of heads for multi-head SSD. |
int NumLayers { get; set; } |
Gets or sets the number of Mamba-2 layers. |
int StateDimension { get; set; } |
Gets or sets the state dimension per head. |