RWKVForecastingOptions<T>
Options & Configuration · AiDotNet.Models.Options
Configuration options for RWKV-based time series forecasting.
Inheritance: ModelOptions → RegressionOptions<T> → TimeSeriesRegressionOptions<T> → RWKVForecastingOptions<T>
For Beginners
RWKV combines the best of both worlds:
Key Properties:
- Linear Complexity: O(n) for training and inference (vs O(n^2) for Transformers)
- Constant Memory: O(1) per-token generation memory
- Parallel Training: Can be computed as a convolution for efficient parallel training
- Multi-head: Multiple attention heads for better capacity
Architecture:
- Time mixing: WKV attention mechanism with learned decay
- Channel mixing: FFN with gating
- Residual connections and layer normalization
How It Works
RWKV (Receptance Weighted Key Value) is a linear-complexity sequence model that combines the efficient training parallelism of Transformers with the constant-memory inference of RNNs. This options class configures an RWKV model for time series forecasting tasks.
Reference: Peng et al., “RWKV: Reinventing RNNs for the Transformer Era”, 2023.
Constructors
| Constructor | Summary |
|---|---|
RWKVForecastingOptions() |
Initializes a new instance with default values. |
RWKVForecastingOptions(RWKVForecastingOptions<T> other) |
Initializes a new instance by copying from another instance. |
Properties
| Property | Summary |
|---|---|
bool UseReversibleNormalization { get; set; } |
Gets or sets whether reversible instance normalization (RevIN) is applied around the forecast. |
double AdamBeta1 { get; set; } |
Gets or sets Adam’s first-moment decay. |
double AdamBeta2 { get; set; } |
Gets or sets Adam’s second-moment decay. |
double AdamEpsilon { get; set; } |
Gets or sets Adam’s numerical-stability epsilon. |
double DropoutRate { get; set; } |
Gets or sets the dropout rate for regularization. |
double GlobalIclrMultiplier { get; set; } |
Gets or sets RWKV-7’s “Global ICLR Multiplier” c in the state transition A_t = diag(w_t) - c * kappaHat^T(a_t (*) kappaHat). |
double LearningRate { get; set; } |
Gets or sets Adam’s initial learning rate. |
double RevInEpsilon { get; set; } |
Gets or sets the variance floor used by RevIN’s per-instance standard deviation, std = sqrt(var + eps). |
int ContextLength { get; set; } |
Gets or sets the context length (input sequence length). |
int ForecastHorizon { get; set; } |
Gets or sets the forecast horizon (prediction length). |
int ModelDimension { get; set; } |
Gets or sets the model dimension (d_model). |
int NumHeads { get; set; } |
Gets or sets the number of RWKV heads. |
int NumLayers { get; set; } |
Gets or sets the number of RWKV layers. |