Mamba2Options
Options & Configuration · AiDotNet.NeuralNetworks.Options
Configures the dimensions, state width, and head count of the Mamba-2 language model.
Inheritance: ModelOptions → NeuralNetworkOptions → ModelHyperparameterOptions → SequenceModelOptions → Mamba2Options
For Beginners
Mamba-2 keeps a running summary instead of retaining an attention score for every earlier token. Its state width controls the size of that memory; its head count divides the internal computation into groups.
How It Works
Dao and Gu, Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Duality (2024), introduce the structured state-space duality formulation used by Mamba-2. Released configurations range from 130 million to 2.7 billion parameters; the defaults are the smallest of them, Mamba-2 130M.
Constructors
| Constructor | Summary |
|---|---|
Mamba2Options() |
Initializes a new instance of the Mamba2Options class carrying this model’s shipped defaults. |
Mamba2Options(Mamba2Options other) |
Initializes an instance by copying every declared and inherited setting. |