RWKV4Options

Options & Configuration · AiDotNet.NeuralNetworks.Options

Configuration options for the RWKV4LanguageModel.

Inheritance: ModelOptions → NeuralNetworkOptions → ModelHyperparameterOptions → SequenceModelOptions → RWKV4Options

For Beginners

RWKV-4 keeps a compact recurrent memory while training with parallel token operations. Width and layer count trade memory and compute for capacity.

How It Works

Peng et al., RWKV: Reinventing RNNs for the Transformer Era (2023), Table 2, report 169M (width 768, 12 layers), 1.5B (2048, 24), 7B (4096, 32), and 14B (5120, 40) configurations. Those are published sizing examples, not the defaults below: this library retains its existing width 256 and four layers for compatibility. Matching size alone does not reproduce pretrained weights or the full training recipe.

Constructors

Constructor Summary
RWKV4Options() Initializes a new instance of the RWKV4Options class carrying this model’s shipped defaults.
RWKV4Options(RWKV4Options other) Initializes an instance by copying every declared and inherited setting.

Methods

Method Summary
void Validate() Throws if a value this model requires has been left unset or is not positive.