NonStationaryTransformer<T>

Models & Types · AiDotNet.Finance.Forecasting.Transformers

Non-stationary Transformer neural network for time series forecasting with changing statistics.

Inheritance: NeuralNetworkBase<T> → FinancialModelBase<T> → ForecastingModelBase<T> → NonStationaryTransformer<T>

For Beginners

Real-world time series data often has changing statistical properties:

  • Stock prices trend up or down over time
  • Sales data has seasonal patterns that change intensity
  • Energy consumption varies with economic conditions

Traditional approaches normalize this data (make it “stationary”), but this can lose important information. Non-stationary Transformer:

  1. Series Stationarization: Normalizes data for better attention computation
  2. De-stationary Attention: Learns to restore original statistical properties

This gives you the best of both worlds - good pattern recognition AND preserved data characteristics.

How It Works

Non-stationary Transformer addresses the over-stationarization problem in time series forecasting by proposing Series Stationarization and De-stationary Attention mechanisms that preserve original data characteristics while benefiting from attention’s pattern recognition.

Reference: Liu et al., “Non-stationary Transformers: Exploring the Stationarity in Time Series Forecasting”, NeurIPS 2022. https://arxiv.org/abs/2205.14415

Example

// Create a Non-stationary Transformer for data with changing statistical properties
// Preserves non-stationary characteristics while using attention for pattern recognition
var architecture = new NeuralNetworkArchitecture<double>(
    inputType: InputType.OneDimensional,
    taskType: NeuralNetworkTaskType.Regression,
    inputHeight: 96, inputWidth: 7, inputDepth: 1, outputSize: 24);
            
// Training mode with de-stationary attention mechanism
var model = new NonStationaryTransformer<double>(architecture);
            
// ONNX inference mode with pre-trained model
var onnxModel = new NonStationaryTransformer<double>(architecture, "ns_transformer.onnx");

Constructors

Constructor Summary
NonStationaryTransformer(NeuralNetworkArchitecture<T> architecture, NonStationaryTransformerOptions<T> options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer, ILossFunction<T> lossFunction) Creates a new Non-stationary Transformer instance for native training mode.
NonStationaryTransformer(NeuralNetworkArchitecture<T> architecture, string onnxModelPath, NonStationaryTransformerOptions<T> options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer, ILossFunction<T> lossFunction) Creates a new Non-stationary Transformer instance for ONNX inference mode.

Properties

Property Summary
bool IsChannelIndependent { get; } Gets whether the model processes channels independently.
bool SupportsTraining { get; } Gets whether this network supports training.
bool UseNativeMode { get; } Gets whether the model uses native mode (true) or ONNX mode (false).
int NumFeatures { get; } Gets the number of input features.
int PatchSize { get; } Gets the patch size.
int PredictionHorizon { get; } Gets the prediction horizon.
int SequenceLength { get; } Gets the input sequence length.
int Stride { get; } Gets the stride.

Methods

Method Summary
Dictionary<string, T> Evaluate(Tensor<T> testData, Tensor<T> testLabels) Evaluates the model on a test dataset.
Dictionary<string, T> GetFinancialMetrics() Gets financial metrics specific to the model.
ModelMetadata<T> GetModelMetadata() Gets metadata about the model.
Tensor<T> ApplyInstanceNormalization(Tensor<T> input) Applies instance normalization (RevIN) to the input.
Tensor<T> AutoregressiveForecast(Tensor<T> input, int steps) Generates multi-step forecasts using autoregressive prediction.
Tensor<T> ForwardForTraining(Tensor<T> input) Tape-aware training forward.