RWKVForecaster<T>
Models & Types · AiDotNet.Finance.Forecasting.StateSpace
RWKV (Receptance Weighted Key Value) implementation for time series forecasting.
Inheritance: NeuralNetworkBase<T> → FinancialModelBase<T> → ForecastingModelBase<T> → RWKVForecaster<T>
For Beginners
RWKV is a unique architecture for sequence modeling:
The Key Innovation: RWKV can be computed two ways:
- As a parallel operation during training (like a Transformer) — fast on GPUs
- As a recurrence during inference (like an RNN) — constant memory per token
Architecture Components:
- Time mixing: WKV (Weighted Key Value) attention with learned exponential decay
- Channel mixing: Feed-forward network with gating mechanism
- Token shift: Efficiently mixes current and previous token information
For Time Series:
- Linear complexity enables processing very long historical windows
- Constant memory during autoregressive forecasting
- Multi-head structure captures diverse temporal patterns
How It Works
RWKV combines the efficient parallelizable training of Transformers with the efficient inference of RNNs, achieving linear complexity for both training and inference. This forecasting variant uses stacked RWKV layers for time series prediction.
Reference: Peng et al., “RWKV: Reinventing RNNs for the Transformer Era”, 2023.
Example
var architecture = new NeuralNetworkArchitecture<double>(
inputType: InputType.OneDimensional,
taskType: NeuralNetworkTaskType.Regression,
inputHeight: 512, inputWidth: 1, inputDepth: 1, outputSize: 24);
var model = new RWKVForecaster<double>(architecture);
var onnxModel = new RWKVForecaster<double>(architecture, "rwkv.onnx");
Constructors
| Constructor | Summary |
|---|---|
RWKVForecaster(NeuralNetworkArchitecture<T> architecture, RWKVForecastingOptions<T> options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer, ILossFunction<T> lossFunction) |
Initializes a new instance in native mode for training. |
RWKVForecaster(NeuralNetworkArchitecture<T> architecture, string onnxModelPath, RWKVForecastingOptions<T> options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer, ILossFunction<T> lossFunction) |
Initializes a new instance using an ONNX pretrained model. |
Properties
| Property | Summary |
|---|---|
bool IsChannelIndependent { get; } |
|
bool SupportsTraining { get; } |
|
bool UseNativeMode { get; } |
|
int NumFeatures { get; } |
|
int NumHeads { get; } |
Gets the number of RWKV heads. |
int PatchSize { get; } |
|
int PredictionHorizon { get; } |
|
int SequenceLength { get; } |
|
int Stride { get; } |
Methods
| Method | Summary |
|---|---|
Dictionary<string, T> Evaluate(Tensor<T> predictions, Tensor<T> actuals) |
|
Dictionary<string, T> GetFinancialMetrics() |
|
Dictionary<string, Tensor<T>> GetNamedLayerActivations(Tensor<T> input) |
Captures the per-layer activations along the model’s real forward path. |
ModelMetadata<T> GetModelMetadata() |
|
Tensor<T> ApplyInstanceNormalization(Tensor<T> input) |
|
Tensor<T> AutoregressiveForecast(Tensor<T> input, int steps) |