Mamba2<T>
Models & Types · AiDotNet.Finance.Forecasting.StateSpace
Mamba-2 (State Space Duality) implementation for time series forecasting.
Inheritance: NeuralNetworkBase<T> → FinancialModelBase<T> → ForecastingModelBase<T> → Mamba2<T>
For Beginners
Mamba-2 is the next generation of the Mamba architecture:
Key Improvements over Mamba-1:
- SSD Algorithm: Uses matrix multiply instead of associative scan — much faster on GPUs
- Multi-head Structure: Like multi-head attention, enabling better capacity per parameter
- Chunk-wise Processing: Processes sequences in chunks for better hardware utilization
- 2-8x Faster Training: Due to better hardware mapping
For Time Series:
- Efficient handling of long historical windows
- Multi-head captures different temporal patterns simultaneously
- Linear complexity enables real-time forecasting on long sequences
How It Works
Mamba-2 improves upon Mamba by discovering the connection between selective state space models and structured masked attention (State Space Duality). This enables a more efficient SSD algorithm using matrix multiplications and multi-head structure, achieving 2-8x faster training.
Reference: Dao and Gu, “Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Duality”, 2024.
Example
var architecture = new NeuralNetworkArchitecture<double>(
inputType: InputType.OneDimensional,
taskType: NeuralNetworkTaskType.Regression,
inputHeight: 512, inputWidth: 1, inputDepth: 1, outputSize: 24);
var model = new Mamba2<double>(architecture);
var onnxModel = new Mamba2<double>(architecture, "mamba2.onnx");
Constructors
| Constructor | Summary |
|---|---|
Mamba2(NeuralNetworkArchitecture<T> architecture, Mamba2Options<T> options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer, ILossFunction<T> lossFunction) |
Initializes a new instance in native mode for training. |
Mamba2(NeuralNetworkArchitecture<T> architecture, string onnxModelPath, Mamba2Options<T> options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer, ILossFunction<T> lossFunction) |
Initializes a new instance using an ONNX pretrained model. |
Properties
| Property | Summary |
|---|---|
bool IsChannelIndependent { get; } |
|
bool SupportsTraining { get; } |
|
bool UseNativeMode { get; } |
|
int ChunkSize { get; } |
Gets the chunk size for SSD computation. |
int NumFeatures { get; } |
|
int NumHeads { get; } |
Gets the number of heads for multi-head SSD. |
int PatchSize { get; } |
|
int PredictionHorizon { get; } |
|
int SequenceLength { get; } |
|
int StateDimension { get; } |
Gets the state dimension per head. |
int Stride { get; } |
Methods
| Method | Summary |
|---|---|
Dictionary<string, T> Evaluate(Tensor<T> predictions, Tensor<T> actuals) |
|
Dictionary<string, T> GetFinancialMetrics() |
|
Dictionary<string, Tensor<T>> GetNamedLayerActivations(Tensor<T> input) |
|
ModelMetadata<T> GetModelMetadata() |
|
Tensor<T> ApplyInstanceNormalization(Tensor<T> input) |
|
Tensor<T> AutoregressiveForecast(Tensor<T> input, int steps) |