NonStationaryTransformer<T>
Models & Types · AiDotNet.Finance.Forecasting.Transformers
Non-stationary Transformer neural network for time series forecasting with changing statistics.
Inheritance: NeuralNetworkBase<T> → FinancialModelBase<T> → ForecastingModelBase<T> → NonStationaryTransformer<T>
For Beginners
Real-world time series data often has changing statistical properties:
- Stock prices trend up or down over time
- Sales data has seasonal patterns that change intensity
- Energy consumption varies with economic conditions
Traditional approaches normalize this data (make it “stationary”), but this can lose important information. Non-stationary Transformer:
- Series Stationarization: Normalizes data for better attention computation
- De-stationary Attention: Learns to restore original statistical properties
This gives you the best of both worlds - good pattern recognition AND preserved data characteristics.
How It Works
Non-stationary Transformer addresses the over-stationarization problem in time series forecasting by proposing Series Stationarization and De-stationary Attention mechanisms that preserve original data characteristics while benefiting from attention’s pattern recognition.
Reference: Liu et al., “Non-stationary Transformers: Exploring the Stationarity
in Time Series Forecasting”, NeurIPS 2022. https://arxiv.org/abs/2205.14415
Example
// Create a Non-stationary Transformer for data with changing statistical properties
// Preserves non-stationary characteristics while using attention for pattern recognition
var architecture = new NeuralNetworkArchitecture<double>(
inputType: InputType.OneDimensional,
taskType: NeuralNetworkTaskType.Regression,
inputHeight: 96, inputWidth: 7, inputDepth: 1, outputSize: 24);
// Training mode with de-stationary attention mechanism
var model = new NonStationaryTransformer<double>(architecture);
// ONNX inference mode with pre-trained model
var onnxModel = new NonStationaryTransformer<double>(architecture, "ns_transformer.onnx");
Constructors
| Constructor | Summary |
|---|---|
NonStationaryTransformer(NeuralNetworkArchitecture<T> architecture, NonStationaryTransformerOptions<T> options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer, ILossFunction<T> lossFunction) |
Creates a new Non-stationary Transformer instance for native training mode. |
NonStationaryTransformer(NeuralNetworkArchitecture<T> architecture, string onnxModelPath, NonStationaryTransformerOptions<T> options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer, ILossFunction<T> lossFunction) |
Creates a new Non-stationary Transformer instance for ONNX inference mode. |
Properties
| Property | Summary |
|---|---|
bool IsChannelIndependent { get; } |
Gets whether the model processes channels independently. |
bool SupportsTraining { get; } |
Gets whether this network supports training. |
bool UseNativeMode { get; } |
Gets whether the model uses native mode (true) or ONNX mode (false). |
int NumFeatures { get; } |
Gets the number of input features. |
int PatchSize { get; } |
Gets the patch size. |
int PredictionHorizon { get; } |
Gets the prediction horizon. |
int SequenceLength { get; } |
Gets the input sequence length. |
int Stride { get; } |
Gets the stride. |
Methods
| Method | Summary |
|---|---|
Dictionary<string, T> Evaluate(Tensor<T> testData, Tensor<T> testLabels) |
Evaluates the model on a test dataset. |
Dictionary<string, T> GetFinancialMetrics() |
Gets financial metrics specific to the model. |
ModelMetadata<T> GetModelMetadata() |
Gets metadata about the model. |
Tensor<T> ApplyInstanceNormalization(Tensor<T> input) |
Applies instance normalization (RevIN) to the input. |
Tensor<T> AutoregressiveForecast(Tensor<T> input, int steps) |
Generates multi-step forecasts using autoregressive prediction. |
Tensor<T> ForwardForTraining(Tensor<T> input) |
Tape-aware training forward. |