S4<T>
Models & Types · AiDotNet.Finance.Forecasting.StateSpace
S4 (Structured State Space Sequence Model) for time series forecasting.
Inheritance: NeuralNetworkBase<T> → FinancialModelBase<T> → ForecastingModelBase<T> → S4<T>
For Beginners
S4 is a breakthrough model that showed state space models can match transformers on long-range sequence tasks:
The Core Idea: S4 uses a “structured” state space model where:
- The A matrix is initialized using HiPPO (optimal for compressing history)
- A is decomposed as diagonal + low-rank for efficiency (DPLR)
- The computation is done as a convolution using FFT (O(n log n))
Why S4 Works:
- HiPPO initialization: The state optimally compresses the input history
- DPLR structure: Allows efficient computation while keeping HiPPO’s benefits
- FFT convolution: Converts sequential computation to parallel
The Math (simplified): State Space Model: x’(t) = Ax(t) + Bu(t), y(t) = Cx(t) + Du(t)
- A: State transition (N x N HiPPO matrix, DPLR decomposed)
- B: Input projection (N x 1)
- C: Output projection (1 x N)
- D: Direct feedthrough (skip connection)
Discretized for sequences: x_k = A_bar * x_{k-1} + B_bar * u_k Computed as convolution: y = K * u where K is the SSM kernel
Key Benefits:
- O(n log n) complexity via FFT (vs O(n^2) for attention)
- Handles very long sequences (16K+ tokens)
- Foundation for modern SSMs (Mamba, H3, etc.)
- Strong performance on Long Range Arena benchmark
How It Works
S4 is a foundational state space model that achieves near-linear complexity through structured parameterization of the state transition matrix using the HiPPO framework.
Reference: Gu et al., “Efficiently Modeling Long Sequences with Structured State Spaces”, 2022.
https://arxiv.org/abs/2111.00396
Example
var architecture = new NeuralNetworkArchitecture<double>(
inputType: InputType.OneDimensional,
taskType: NeuralNetworkTaskType.Regression,
inputHeight: 256, inputWidth: 1, inputDepth: 1, outputSize: 24);
var model = new S4<double>(architecture);
var onnxModel = new S4<double>(architecture, "s4.onnx");
Constructors
| Constructor | Summary |
|---|---|
S4(NeuralNetworkArchitecture<T> architecture, S4Options<T> options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer, ILossFunction<T> lossFunction) |
Initializes a new instance of the S4 model in native mode for training. |
S4(NeuralNetworkArchitecture<T> architecture, string onnxModelPath, S4Options<T> options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer, ILossFunction<T> lossFunction) |
Initializes a new instance of the S4 model in ONNX mode for inference. |
Properties
| Property | Summary |
|---|---|
bool IsChannelIndependent { get; } |
|
bool SupportsTraining { get; } |
Gets whether the model supports training (native mode only). |
bool UseLowRankCorrection { get; } |
Gets whether low-rank correction is used in DPLR decomposition. |
bool UseNativeMode { get; } |
|
int ContextLength { get; } |
Gets the input context length for the model. |
int ForecastHorizon { get; } |
Gets the forecast horizon (number of future steps to predict). |
int NumFeatures { get; } |
|
int PatchSize { get; } |
|
int PredictionHorizon { get; } |
|
int SequenceLength { get; } |
|
int StateDimension { get; } |
Gets the state dimension of the SSM. |
int Stride { get; } |
|
string HippoMethod { get; } |
Gets the HiPPO method used for state initialization. |
Methods
| Method | Summary |
|---|---|
Dictionary<string, T> Evaluate(Tensor<T> predictions, Tensor<T> actuals) |
Evaluates forecast quality against actual values. |
Dictionary<string, T> GetFinancialMetrics() |
Gets financial-specific metrics about the model. |
ModelMetadata<T> GetModelMetadata() |
Gets metadata about the S4 model. |
Tensor<T> ApplyInstanceNormalization(Tensor<T> input) |
Applies instance normalization to the input. |
Tensor<T> AutoregressiveForecast(Tensor<T> input, int steps) |
Performs autoregressive forecasting step by step. |
Tensor<T> Forward(Tensor<T> input) |
Performs the forward pass through all layers. |
ValueTuple<Tensor<T>, Tensor<T>, Tensor<T>> ForecastWithIntervals(Tensor<T> input, double confidenceLevel) |
Generates forecasts with prediction intervals for uncertainty quantification. |