SambaOptions

Options & Configuration · AiDotNet.NeuralNetworks.Options

Configures the dimensions and attention spacing of the Samba language model.

Inheritance: ModelOptions → NeuralNetworkOptions → ModelHyperparameterOptions → SequenceModelOptions → SambaOptions

For Beginners

Samba combines a running memory for older information with attention to recent tokens. Its attention interval controls how often recurrent blocks are interleaved with attention blocks.

How It Works

Ren et al., Samba: Simple Hybrid State Space Models for Efficient Unlimited Context Language Modeling (2024), combine Mamba with sliding-window attention and evaluate models up to 3.8 billion parameters. The existing 256-wide, eight-layer defaults are a small library configuration, not the published 3.8B checkpoint. These settings do not independently specify the paper’s entire training recipe or attention-window policy.

Constructors

Constructor Summary
SambaOptions() Initializes a new instance of the SambaOptions class carrying this model’s shipped defaults.
SambaOptions(SambaOptions other) Initializes an instance by copying every declared and inherited setting.

Methods

Method Summary
void Validate() Throws if a required model dimension or consumed training setting is invalid.