Informer<T>

Models & Types · AiDotNet.Finance.Forecasting.Transformers

Informer (Efficient Transformer for Long Sequence Forecasting) neural network.

Inheritance: NeuralNetworkBase<T> → FinancialModelBase<T> → ForecastingModelBase<T> → Informer<T>

For Beginners

In regular transformers, every position looks at every other position (O(n²) complexity), which is slow for long sequences. Informer is clever - it figures out which positions are most “active” (have high variance in their attention scores) and only computes attention for those.

Key innovations:

  • ProbSparse Attention: Only compute attention for top-k important queries
  • Self-attention Distilling: Progressively reduce sequence length with max-pooling
  • Generative Decoder: Predict all future values at once, not step-by-step

How It Works

Informer was the first transformer specifically designed for long sequence time series forecasting. It achieved AAAI 2021 Best Paper for its innovations in efficient attention.

Reference: Zhou et al., “Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting”, AAAI 2021 (Best Paper). https://arxiv.org/abs/2012.07436

The network is the official code’s (InformerNetwork, shared with InformerModel and checked against Informer2020), forecasting every input series jointly. Standardize the data with training statistics first, as the reference does. The temporal embedding needs timestamps, which this interface does not carry, so it is not used.

Example

// Create an Informer for efficient long-sequence time series forecasting
// Uses ProbSparse attention for O(L log L) complexity (AAAI 2021 Best Paper)
var architecture = new NeuralNetworkArchitecture<double>(
    inputType: InputType.OneDimensional,
    taskType: NeuralNetworkTaskType.Regression,
    inputSize: 7, outputSize: 7);   // seven series (ETTh1), forecast jointly
            
// Training mode with ProbSparse self-attention and distilling: [batch, 96, 7] in, [batch, 24, 7] out
var model = new Informer<double>(architecture);
            
// ONNX inference mode with pre-trained model
var onnxModel = new Informer<double>(architecture, "informer_etth1.onnx");

Constructors

Constructor Summary
Informer() Creates an Informer with the reference’s defaults for a single series.
Informer(NeuralNetworkArchitecture<T> architecture, InformerOptions<T> options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer, ILossFunction<T> lossFunction) Creates a trainable Informer.
Informer(NeuralNetworkArchitecture<T> architecture, string onnxModelPath, InformerOptions<T> options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer, ILossFunction<T> lossFunction) Creates an Informer that runs a pre-trained ONNX model.

Properties

Property Summary
bool IsChannelIndependent { get; }
bool SupportsTraining { get; }
bool UseNativeMode { get; }
int NumFeatures { get; }
int PatchSize { get; }
int PredictionHorizon { get; }
int SequenceLength { get; }
int Stride { get; }

Methods

Method Summary
Dictionary<string, T> Evaluate(Tensor<T> inputs, Tensor<T> targets)
Dictionary<string, T> GetFinancialMetrics()
Dictionary<string, Tensor<T>> GetNamedLayerActivations(Tensor<T> input)
ModelMetadata<T> GetModelMetadata()
Tensor<T> ApplyInstanceNormalization(Tensor<T> input)
Tensor<T> AutoregressiveForecast(Tensor<T> input, int steps)