Autoformer<T>

Models & Types · AiDotNet.Finance.Forecasting.Transformers

Autoformer (Decomposition Transformers with Auto-Correlation) neural network for time series forecasting.

Inheritance: NeuralNetworkBase<T> → FinancialModelBase<T> → ForecastingModelBase<T> → Autoformer<T>

For Beginners

Autoformer is like a music analyst who finds repeating patterns by checking if the melody matches itself at different time delays. Instead of comparing every note to every other note (O(L²)), it uses FFT to find these patterns in O(L log L) time.

Key innovations:

  • Auto-Correlation: Finds periodic patterns using FFT instead of attention
  • Series Decomposition: Separates trend from seasonal patterns at each layer
  • Progressive Decomposition: Decomposition happens multiple times for accuracy

How It Works

Autoformer replaces the traditional attention mechanism with auto-correlation, which finds period-based dependencies efficiently using FFT. It also progressively decomposes the series into trend and seasonal components at each layer.

Reference: Wu et al., “Autoformer: Decomposition Transformers with Auto-Correlation”, NeurIPS 2021. https://arxiv.org/abs/2106.13008

The network is the official code’s (AutoformerNetwork, shared with AutoformerModel and checked against thuml/Autoformer), forecasting every input series jointly. Standardize the data with training statistics first, as the reference does. The temporal embedding needs timestamps, which this interface does not carry, so it is not used.

Example

// Create an Autoformer for long-term financial time series forecasting
// with auto-correlation attention and series decomposition
var architecture = new NeuralNetworkArchitecture<double>(
    inputType: InputType.OneDimensional,
    taskType: NeuralNetworkTaskType.Regression,
    inputSize: 7, outputSize: 7);   // seven series (ETTh1), forecast jointly
            
// Training mode: [batch, 96, 7] in, [batch, 96, 7] out
var model = new Autoformer<double>(architecture);
            
// ONNX inference mode with pre-trained model
var onnxModel = new Autoformer<double>(architecture, "autoformer_etth1.onnx");

Constructors

Constructor Summary
Autoformer() Creates an Autoformer with the reference’s defaults for a single series.
Autoformer(NeuralNetworkArchitecture<T> architecture, AutoformerOptions<T> options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer, ILossFunction<T> lossFunction) Creates a trainable Autoformer.
Autoformer(NeuralNetworkArchitecture<T> architecture, string onnxModelPath, AutoformerOptions<T> options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer, ILossFunction<T> lossFunction) Creates an Autoformer that runs a pre-trained ONNX model.

Properties

Property Summary
bool IsChannelIndependent { get; }
bool SupportsTraining { get; }
bool UseNativeMode { get; }
int NumFeatures { get; }
int PatchSize { get; }
int PredictionHorizon { get; }
int SequenceLength { get; }
int Stride { get; }

Methods

Method Summary
Dictionary<string, T> Evaluate(Tensor<T> inputs, Tensor<T> targets)
Dictionary<string, T> GetFinancialMetrics()
Dictionary<string, Tensor<T>> GetNamedLayerActivations(Tensor<T> input)
ModelMetadata<T> GetModelMetadata()
Tensor<T> ApplyInstanceNormalization(Tensor<T> input)
Tensor<T> AutoregressiveForecast(Tensor<T> input, int steps)