JambaOptions

Options & Configuration · AiDotNet.NeuralNetworks.Options

Configures the dimensions and attention spacing of the Jamba language model.

Inheritance: ModelOptions → NeuralNetworkOptions → ModelHyperparameterOptions → SequenceModelOptions → JambaOptions

For Beginners

Jamba combines a compact recurrent memory with attention, which can look back at individual tokens. AttentionInterval controls how frequently this model inserts an attention block.

How It Works

Lieber et al., Jamba: A Hybrid Transformer-Mamba Language Model (2024), describe interleaved Transformer/Mamba layers and mixture-of-experts feed-forward layers. The original release has 52 billion total parameters and 12 billion active per token. These options expose the library’s sequence dimensions, not the paper’s complete MoE configuration. The existing 256-wide, eight-layer defaults remain small library defaults, not a reproduction of that checkpoint.

Constructors

Constructor Summary
JambaOptions() Initializes a new instance of the JambaOptions class carrying this model’s shipped defaults.
JambaOptions(JambaOptions other) Initializes an instance by copying every declared and inherited setting.