Informer<T>
Models & Types · AiDotNet.Finance.Forecasting.Transformers
Informer (Efficient Transformer for Long Sequence Forecasting) neural network.
Inheritance: NeuralNetworkBase<T> → FinancialModelBase<T> → ForecastingModelBase<T> → Informer<T>
For Beginners
In regular transformers, every position looks at every other position (O(n²) complexity), which is slow for long sequences. Informer is clever - it figures out which positions are most “active” (have high variance in their attention scores) and only computes attention for those.
Key innovations:
- ProbSparse Attention: Only compute attention for top-k important queries
- Self-attention Distilling: Progressively reduce sequence length with max-pooling
- Generative Decoder: Predict all future values at once, not step-by-step
How It Works
Informer was the first transformer specifically designed for long sequence time series forecasting. It achieved AAAI 2021 Best Paper for its innovations in efficient attention.
Reference: Zhou et al., “Informer: Beyond Efficient Transformer for Long Sequence
Time-Series Forecasting”, AAAI 2021 (Best Paper). https://arxiv.org/abs/2012.07436
The network is the official code’s (InformerNetwork, shared with
InformerModel and checked against Informer2020), forecasting every input
series jointly. Standardize the data with training statistics first, as the reference does. The temporal
embedding needs timestamps, which this interface does not carry, so it is not used.
Example
// Create an Informer for efficient long-sequence time series forecasting
// Uses ProbSparse attention for O(L log L) complexity (AAAI 2021 Best Paper)
var architecture = new NeuralNetworkArchitecture<double>(
inputType: InputType.OneDimensional,
taskType: NeuralNetworkTaskType.Regression,
inputSize: 7, outputSize: 7); // seven series (ETTh1), forecast jointly
// Training mode with ProbSparse self-attention and distilling: [batch, 96, 7] in, [batch, 24, 7] out
var model = new Informer<double>(architecture);
// ONNX inference mode with pre-trained model
var onnxModel = new Informer<double>(architecture, "informer_etth1.onnx");
Constructors
| Constructor | Summary |
|---|---|
Informer() |
Creates an Informer with the reference’s defaults for a single series. |
Informer(NeuralNetworkArchitecture<T> architecture, InformerOptions<T> options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer, ILossFunction<T> lossFunction) |
Creates a trainable Informer. |
Informer(NeuralNetworkArchitecture<T> architecture, string onnxModelPath, InformerOptions<T> options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer, ILossFunction<T> lossFunction) |
Creates an Informer that runs a pre-trained ONNX model. |
Properties
| Property | Summary |
|---|---|
bool IsChannelIndependent { get; } |
|
bool SupportsTraining { get; } |
|
bool UseNativeMode { get; } |
|
int NumFeatures { get; } |
|
int PatchSize { get; } |
|
int PredictionHorizon { get; } |
|
int SequenceLength { get; } |
|
int Stride { get; } |
Methods
| Method | Summary |
|---|---|
Dictionary<string, T> Evaluate(Tensor<T> inputs, Tensor<T> targets) |
|
Dictionary<string, T> GetFinancialMetrics() |
|
Dictionary<string, Tensor<T>> GetNamedLayerActivations(Tensor<T> input) |
|
ModelMetadata<T> GetModelMetadata() |
|
Tensor<T> ApplyInstanceNormalization(Tensor<T> input) |
|
Tensor<T> AutoregressiveForecast(Tensor<T> input, int steps) |