RWKV4Options
Options & Configuration · AiDotNet.NeuralNetworks.Options
Configuration options for the RWKV4LanguageModel.
Inheritance: ModelOptions → NeuralNetworkOptions → ModelHyperparameterOptions → SequenceModelOptions → RWKV4Options
For Beginners
RWKV-4 keeps a compact recurrent memory while training with parallel token operations. Width and layer count trade memory and compute for capacity.
How It Works
Peng et al., RWKV: Reinventing RNNs for the Transformer Era (2023), Table 2, report 169M (width 768, 12 layers), 1.5B (2048, 24), 7B (4096, 32), and 14B (5120, 40) configurations. Those are published sizing examples, not the defaults below: this library retains its existing width 256 and four layers for compatibility. Matching size alone does not reproduce pretrained weights or the full training recipe.
Constructors
| Constructor | Summary |
|---|---|
RWKV4Options() |
Initializes a new instance of the RWKV4Options class carrying this model’s shipped defaults. |
RWKV4Options(RWKV4Options other) |
Initializes an instance by copying every declared and inherited setting. |
Methods
| Method | Summary |
|---|---|
void Validate() |
Throws if a value this model requires has been left unset or is not positive. |