Mamba2<T>

Models & Types · AiDotNet.Finance.Forecasting.StateSpace

Mamba-2 (State Space Duality) implementation for time series forecasting.

Inheritance: NeuralNetworkBase<T> → FinancialModelBase<T> → ForecastingModelBase<T> → Mamba2<T>

For Beginners

Mamba-2 is the next generation of the Mamba architecture:

Key Improvements over Mamba-1:

  1. SSD Algorithm: Uses matrix multiply instead of associative scan — much faster on GPUs
  2. Multi-head Structure: Like multi-head attention, enabling better capacity per parameter
  3. Chunk-wise Processing: Processes sequences in chunks for better hardware utilization
  4. 2-8x Faster Training: Due to better hardware mapping

For Time Series:

  • Efficient handling of long historical windows
  • Multi-head captures different temporal patterns simultaneously
  • Linear complexity enables real-time forecasting on long sequences

How It Works

Mamba-2 improves upon Mamba by discovering the connection between selective state space models and structured masked attention (State Space Duality). This enables a more efficient SSD algorithm using matrix multiplications and multi-head structure, achieving 2-8x faster training.

Reference: Dao and Gu, “Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Duality”, 2024.

Example

var architecture = new NeuralNetworkArchitecture<double>(
    inputType: InputType.OneDimensional,
    taskType: NeuralNetworkTaskType.Regression,
    inputHeight: 512, inputWidth: 1, inputDepth: 1, outputSize: 24);
var model = new Mamba2<double>(architecture);
var onnxModel = new Mamba2<double>(architecture, "mamba2.onnx");

Constructors

Constructor Summary
Mamba2(NeuralNetworkArchitecture<T> architecture, Mamba2Options<T> options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer, ILossFunction<T> lossFunction) Initializes a new instance in native mode for training.
Mamba2(NeuralNetworkArchitecture<T> architecture, string onnxModelPath, Mamba2Options<T> options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer, ILossFunction<T> lossFunction) Initializes a new instance using an ONNX pretrained model.

Properties

Property Summary
bool IsChannelIndependent { get; }
bool SupportsTraining { get; }
bool UseNativeMode { get; }
int ChunkSize { get; } Gets the chunk size for SSD computation.
int NumFeatures { get; }
int NumHeads { get; } Gets the number of heads for multi-head SSD.
int PatchSize { get; }
int PredictionHorizon { get; }
int SequenceLength { get; }
int StateDimension { get; } Gets the state dimension per head.
int Stride { get; }

Methods

Method Summary
Dictionary<string, T> Evaluate(Tensor<T> predictions, Tensor<T> actuals)
Dictionary<string, T> GetFinancialMetrics()
Dictionary<string, Tensor<T>> GetNamedLayerActivations(Tensor<T> input)
ModelMetadata<T> GetModelMetadata()
Tensor<T> ApplyInstanceNormalization(Tensor<T> input)
Tensor<T> AutoregressiveForecast(Tensor<T> input, int steps)