ZambaOptions

Options & Configuration · AiDotNet.NeuralNetworks.Options

Configures the dimensions and attention spacing of the Zamba language model.

Inheritance: ModelOptions → NeuralNetworkOptions → ModelHyperparameterOptions → SequenceModelOptions → ZambaOptions

For Beginners

Zamba uses a recurrent backbone for compact memory and adds attention periodically to recover token-level information. Width and depth determine model capacity, while the attention interval controls how often the two mechanisms mix.

How It Works

Glorioso et al., Zamba: A Compact 7B SSM Hybrid Model (2024), describe a seven-billion-parameter Mamba backbone with a shared attention module. These options retain the library’s existing 3712-wide, 76-layer configuration. Matching those dimensions alone does not establish checkpoint, weight-sharing, or training-recipe equivalence.

Constructors

Constructor Summary
ZambaOptions() Initializes a new instance of the ZambaOptions class carrying this model’s shipped defaults.
ZambaOptions(ZambaOptions other) Initializes an instance by copying every declared and inherited setting.

Methods

Method Summary
void Validate() Throws if a required model dimension or consumed training setting is invalid.