ITransformer<T>
Models & Types · AiDotNet.Finance.Forecasting.Transformers
iTransformer (Inverted Transformer) neural network for time series forecasting.
Inheritance: NeuralNetworkBase<T> → FinancialModelBase<T> → ForecastingModelBase<T> → ITransformer<T>
For Beginners
Traditional transformers for time series treat each time step as a “word”. iTransformer flips this - it treats each variable (like price, volume) as a “word”. This way, the attention mechanism learns how different variables relate to each other, which is often more useful for forecasting than just looking at temporal patterns.
How It Works
iTransformer inverts the traditional transformer approach by treating each variable (channel) as a token instead of each time step. This allows the model to learn cross-variable dependencies more effectively through the attention mechanism.
Reference: Liu et al., “iTransformer: Inverted Transformers Are Effective for Time Series Forecasting”,
ICLR 2024. https://arxiv.org/abs/2310.06625
Example
// Create an iTransformer for multivariate forecasting with inverted attention
// Treats each variable as a token instead of each time step for better cross-variable learning
var architecture = new NeuralNetworkArchitecture<double>(
inputType: InputType.OneDimensional,
taskType: NeuralNetworkTaskType.Regression,
inputHeight: 96, inputWidth: 7, inputDepth: 1, outputSize: 24);
// Training mode with inverted variable-as-token attention
var model = new ITransformer<double>(architecture);
// ONNX inference mode with pre-trained model
var onnxModel = new ITransformer<double>(architecture, "itransformer_traffic.onnx");
Constructors
| Constructor | Summary |
|---|---|
ITransformer(NeuralNetworkArchitecture<T> architecture, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer, ILossFunction<T> lossFunction, ITransformerOptions<T> options) |
Creates an iTransformer network using native library layers. |
ITransformer(NeuralNetworkArchitecture<T> architecture, string onnxModelPath, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer, ILossFunction<T> lossFunction, ITransformerOptions<T> options) |
Creates an iTransformer network using a pretrained ONNX model. |
Properties
| Property | Summary |
|---|---|
bool IsChannelIndependent { get; } |
|
bool SupportsTraining { get; } |
|
bool UseNativeMode { get; } |
|
int NumFeatures { get; } |
|
int PatchSize { get; } |
|
int PredictionHorizon { get; } |
|
int SequenceLength { get; } |
|
int Stride { get; } |
Methods
| Method | Summary |
|---|---|
Dictionary<string, T> Evaluate(Tensor<T> inputs, Tensor<T> targets) |
Evaluates the model’s forecasting performance on test data. |
Dictionary<string, T> GetFinancialMetrics() |
Gets financial-specific metrics from the model. |
ModelMetadata<T> GetModelMetadata() |
Gets metadata about the model for serialization and inspection. |
Tensor<T> ApplyInstanceNormalization(Tensor<T> input) |
Applies instance normalization (RevIN) during inference for distribution shift handling. |
Tensor<T> AutoregressiveForecast(Tensor<T> input, int steps) |
Generates multi-step forecasts iteratively (autoregressive forecasting). |