JambaOptions
Options & Configuration · AiDotNet.NeuralNetworks.Options
Configures the dimensions and attention spacing of the Jamba language model.
Inheritance: ModelOptions → NeuralNetworkOptions → ModelHyperparameterOptions → SequenceModelOptions → JambaOptions
For Beginners
Jamba combines a compact recurrent memory with attention,
which can look back at individual tokens. AttentionInterval
controls how frequently this model inserts an attention block.
How It Works
Lieber et al., Jamba: A Hybrid Transformer-Mamba Language Model (2024), describe interleaved Transformer/Mamba layers and mixture-of-experts feed-forward layers. The original release has 52 billion total parameters and 12 billion active per token. These options expose the library’s sequence dimensions, not the paper’s complete MoE configuration. The existing 256-wide, eight-layer defaults remain small library defaults, not a reproduction of that checkpoint.
Constructors
| Constructor | Summary |
|---|---|
JambaOptions() |
Initializes a new instance of the JambaOptions class carrying this model’s shipped defaults. |
JambaOptions(JambaOptions other) |
Initializes an instance by copying every declared and inherited setting. |