MambaOptions<T>
Options & Configuration · AiDotNet.Models.Options
Configuration options for Mamba (Selective State Space Model).
Inheritance: ModelOptions → RegressionOptions<T> → TimeSeriesRegressionOptions<T> → MambaOptions<T>
For Beginners
Mamba is a breakthrough in efficient sequence modeling:
The Key Insight: Transformers have O(n^2) complexity due to attention, which is slow for long sequences. State space models (SSMs) have O(n) complexity but are less expressive. Mamba makes SSM parameters input-dependent (selective), combining the best of both.
How It Works:
- State Space Model: Maintains a hidden state updated recurrently
- Selective Mechanism: Parameters (A, B, C, delta) vary with input
- Hardware-aware Algorithm: Efficient implementation via parallel scan
- Linear Complexity: O(n) time and memory for sequence length n
Architecture:
- Input projection to expanded dimension
- 1D convolution for local context
- Selective SSM core with input-dependent parameters
- Output projection back to model dimension
Advantages:
- Linear time complexity (vs O(n^2) for attention)
- Handles very long sequences efficiently
- Strong performance on language, audio, and time series
- Hardware-efficient implementation
How It Works
Mamba is a selective state space model that achieves linear-time complexity for sequence modeling while maintaining the expressiveness of transformers through input-dependent (selective) state space parameters.
Reference: Gu and Dao, “Mamba: Linear-Time Sequence Modeling with Selective State Spaces”, 2024.
https://arxiv.org/abs/2312.00752
Constructors
| Constructor | Summary |
|---|---|
MambaOptions() |
Initializes a new instance of the MambaOptions class with default values. |
MambaOptions(MambaOptions<T> other) |
Initializes a new instance by copying from another instance. |
Properties
| Property | Summary |
|---|---|
bool UseBidirectional { get; set; } |
Gets or sets whether to use bidirectional processing. |
double DropoutRate { get; set; } |
Gets or sets the dropout rate for regularization. |
int ContextLength { get; set; } |
Gets or sets the context length (input sequence length). |
int ConvKernelSize { get; set; } |
Gets or sets the convolution kernel size. |
int DtRank { get; set; } |
Gets or sets the rank for the delta (dt) projection. |
int ExpandFactor { get; set; } |
Gets or sets the expansion factor for the inner dimension. |
int ForecastHorizon { get; set; } |
Gets or sets the forecast horizon (prediction length). |
int ModelDimension { get; set; } |
Gets or sets the model dimension (d_model). |
int NumLayers { get; set; } |
Gets or sets the number of Mamba layers. |
int StateDimension { get; set; } |
Gets or sets the state dimension (d_state or N). |