SequenceModelOptions
Options & Configuration · AiDotNet.NeuralNetworks.Options
Shared configuration for sequence and language models (Mamba, RWKV, Jamba, Zamba, Griffin, Hawk, xLSTM, RecurrentGemma and relatives).
Inheritance: ModelOptions → NeuralNetworkOptions → ModelHyperparameterOptions → SequenceModelOptions
Known derived types: EagleOptions, FalconMambaOptions, FinchOptions, GLAOptions, GatedDeltaNetOptions, GriffinOptions, HawkOptions, JambaOptions, Mamba2Options, MambaOptions, RWKV4Options, RWKV7Options, RecurrentGemmaOptions, SambaOptions, VisionMambaOptions, XLSTMOptions, Zamba2Options, ZambaOptions
For Beginners
These are the “size” settings of a language model — how many distinct tokens it knows, how wide each internal representation is, how many stacked blocks it has, and how far back it can look. You rarely need to change them: each model’s own options class already ships the sizes from the paper that introduced it.
How It Works
Derived options classes assign their paper’s values in their parameterless constructor. For example:
Properties are non-nullable and defaulted by the derived class rather than nullable
with a GetEffectiveX() fallback. A model’s layer count has no runtime-dependent
best value the way a batch size does — the right value is the paper’s, and it is known
statically. See the “Model Hyperparameter Options” section of the development
guidelines. Infrastructure configuration (telemetry, profiling, AutoML) keeps the
nullable pattern.
Properties
| Property | Summary |
|---|---|
double FfnMultiplier { get; set; } |
Gets or sets the feed-forward width as a multiple of ModelDimension, for models that express it as a ratio (RWKV-7 uses 3.5). |
int AttentionInterval { get; set; } |
Gets or sets how often an attention block is interleaved among recurrent blocks, in hybrid architectures such as Jamba, Samba and Zamba. |
int ExpandFactor { get; set; } |
Gets or sets the inner-projection expansion factor used by state-space blocks. |
int MaxSequenceLength { get; set; } |
Gets or sets the longest sequence, in tokens, the model is configured to process. |
int ModelDimension { get; set; } |
Gets or sets the width of the model’s hidden representation (often written d_model). |
int NumHeads { get; set; } |
Gets or sets the number of attention heads, for models that use attention. |
int NumLayers { get; set; } |
Gets or sets the number of stacked blocks in the model. |
int StateDimension { get; set; } |
Gets or sets the width of the recurrent state, for state-space models. |
int VocabSize { get; set; } |
Gets or sets the number of distinct tokens the model’s embedding table covers. |