Mamba<T>
Models & Types · AiDotNet.Finance.Forecasting.StateSpace
Mamba (Selective State Space Model) implementation for time series forecasting.
Inheritance: NeuralNetworkBase<T> → FinancialModelBase<T> → ForecastingModelBase<T> → Mamba<T>
For Beginners
Mamba is a breakthrough in efficient sequence modeling:
The Key Insight: Transformers have O(n^2) complexity due to attention, which is slow for long sequences. State space models (SSMs) have O(n) complexity but are less expressive. Mamba makes SSM parameters input-dependent (selective), combining the best of both.
How It Works:
- State Space Model: Maintains a hidden state updated recurrently
- Selective Mechanism: Parameters (A, B, C, delta) vary with input
- Hardware-aware Algorithm: Efficient implementation via parallel scan
- Linear Complexity: O(n) time and memory for sequence length n
Advantages:
- Linear time complexity (vs O(n^2) for attention)
- Handles very long sequences efficiently
- Strong performance on language, audio, and time series
- Hardware-efficient implementation
How It Works
Mamba is a selective state space model that achieves linear-time complexity for sequence modeling while maintaining the expressiveness of transformers through input-dependent (selective) state space parameters.
Reference: Gu and Dao, “Mamba: Linear-Time Sequence Modeling with Selective State Spaces”, 2024.
https://arxiv.org/abs/2312.00752
Example
var architecture = new NeuralNetworkArchitecture<double>(
inputType: InputType.OneDimensional,
taskType: NeuralNetworkTaskType.Regression,
inputHeight: 512, inputWidth: 1, inputDepth: 1, outputSize: 24);
var model = new Mamba<double>(architecture);
var onnxModel = new Mamba<double>(architecture, "mamba.onnx");
Constructors
| Constructor | Summary |
|---|---|
Mamba(NeuralNetworkArchitecture<T> architecture, MambaOptions<T> options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer, ILossFunction<T> lossFunction) |
Initializes a new instance in native mode for training. |
Mamba(NeuralNetworkArchitecture<T> architecture, string onnxModelPath, MambaOptions<T> options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer, ILossFunction<T> lossFunction) |
Initializes a new instance using an ONNX pretrained model. |
Properties
| Property | Summary |
|---|---|
bool IsChannelIndependent { get; } |
|
bool SupportsTraining { get; } |
|
bool UseBidirectional { get; } |
Gets whether bidirectional processing is used. |
bool UseNativeMode { get; } |
|
int ExpandFactor { get; } |
Gets the expansion factor. |
int NumFeatures { get; } |
|
int PatchSize { get; } |
|
int PredictionHorizon { get; } |
|
int SequenceLength { get; } |
|
int StateDimension { get; } |
Gets the state dimension of the SSM. |
int Stride { get; } |
Methods
| Method | Summary |
|---|---|
Dictionary<string, T> Evaluate(Tensor<T> predictions, Tensor<T> actuals) |
|
Dictionary<string, T> GetFinancialMetrics() |
|
ModelMetadata<T> GetModelMetadata() |
|
Tensor<T> ApplyInstanceNormalization(Tensor<T> input) |
|
Tensor<T> AutoregressiveForecast(Tensor<T> input, int steps) |