RWKVForecastingOptions<T>

Options & Configuration · AiDotNet.Models.Options

Configuration options for RWKV-based time series forecasting.

Inheritance: ModelOptions → RegressionOptions<T> → TimeSeriesRegressionOptions<T> → RWKVForecastingOptions<T>

For Beginners

RWKV combines the best of both worlds:

Key Properties:

  1. Linear Complexity: O(n) for training and inference (vs O(n^2) for Transformers)
  2. Constant Memory: O(1) per-token generation memory
  3. Parallel Training: Can be computed as a convolution for efficient parallel training
  4. Multi-head: Multiple attention heads for better capacity

Architecture:

  • Time mixing: WKV attention mechanism with learned decay
  • Channel mixing: FFN with gating
  • Residual connections and layer normalization

How It Works

RWKV (Receptance Weighted Key Value) is a linear-complexity sequence model that combines the efficient training parallelism of Transformers with the constant-memory inference of RNNs. This options class configures an RWKV model for time series forecasting tasks.

Reference: Peng et al., “RWKV: Reinventing RNNs for the Transformer Era”, 2023.

Constructors

Constructor Summary
RWKVForecastingOptions() Initializes a new instance with default values.
RWKVForecastingOptions(RWKVForecastingOptions<T> other) Initializes a new instance by copying from another instance.

Properties

Property Summary
bool UseReversibleNormalization { get; set; } Gets or sets whether reversible instance normalization (RevIN) is applied around the forecast.
double AdamBeta1 { get; set; } Gets or sets Adam’s first-moment decay.
double AdamBeta2 { get; set; } Gets or sets Adam’s second-moment decay.
double AdamEpsilon { get; set; } Gets or sets Adam’s numerical-stability epsilon.
double DropoutRate { get; set; } Gets or sets the dropout rate for regularization.
double GlobalIclrMultiplier { get; set; } Gets or sets RWKV-7’s “Global ICLR Multiplier” c in the state transition A_t = diag(w_t) - c * kappaHat^T(a_t (*) kappaHat).
double LearningRate { get; set; } Gets or sets Adam’s initial learning rate.
double RevInEpsilon { get; set; } Gets or sets the variance floor used by RevIN’s per-instance standard deviation, std = sqrt(var + eps).
int ContextLength { get; set; } Gets or sets the context length (input sequence length).
int ForecastHorizon { get; set; } Gets or sets the forecast horizon (prediction length).
int ModelDimension { get; set; } Gets or sets the model dimension (d_model).
int NumHeads { get; set; } Gets or sets the number of RWKV heads.
int NumLayers { get; set; } Gets or sets the number of RWKV layers.