Mamba2Options

Options & Configuration · AiDotNet.NeuralNetworks.Options

Configures the dimensions, state width, and head count of the Mamba-2 language model.

Inheritance: ModelOptions → NeuralNetworkOptions → ModelHyperparameterOptions → SequenceModelOptions → Mamba2Options

For Beginners

Mamba-2 keeps a running summary instead of retaining an attention score for every earlier token. Its state width controls the size of that memory; its head count divides the internal computation into groups.

How It Works

Dao and Gu, Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Duality (2024), introduce the structured state-space duality formulation used by Mamba-2. Released configurations range from 130 million to 2.7 billion parameters; the defaults are the smallest of them, Mamba-2 130M.

Constructors

Constructor Summary
Mamba2Options() Initializes a new instance of the Mamba2Options class carrying this model’s shipped defaults.
Mamba2Options(Mamba2Options other) Initializes an instance by copying every declared and inherited setting.