ITransformerOptions<T>
Options & Configuration · AiDotNet.Models.Options
Configuration options for the iTransformer (Inverted Transformer) model.
Inheritance: ModelOptions → ITransformerOptions<T>
For Beginners
Traditional transformers process time series by treating each time step as a “word” (token). iTransformer flips this - it treats each variable (like price, volume, etc.) as a token. This helps the model learn how different variables relate to each other.
Key settings:
- NumLayers: How many transformer layers to stack. More layers = more capacity.
- NumHeads: Number of attention heads. Each head learns different relationships.
- ModelDimension: Size of internal representations. Larger = more expressive.
- UseChannelAttention: If true, applies attention across channels (the key innovation).
Default values are from the original iTransformer paper and work well for most datasets.
How It Works
iTransformer inverts the traditional transformer approach by treating each variable (channel) as a token instead of each time step. This allows the model to learn cross-variable dependencies more effectively.
Reference: Liu et al., “iTransformer: Inverted Transformers Are Effective for Time Series Forecasting”,
ICLR 2024. https://arxiv.org/abs/2310.06625
Constructors
| Constructor | Summary |
|---|---|
ITransformerOptions() |
Properties
| Property | Summary |
|---|---|
ILossFunction<T> LossFunction { get; set; } |
Gets or sets the loss function for training. |
bool UseChannelAttention { get; set; } |
Gets or sets whether to use channel attention (inverted attention). |
bool UseInstanceNormalization { get; set; } |
Gets or sets whether to use instance normalization (RevIN). |
bool UsePreNorm { get; set; } |
Gets or sets whether to use pre-normalization (LayerNorm before attention). |
double AttentionDropout { get; set; } |
Gets or sets the attention dropout rate. |
double Dropout { get; set; } |
Gets or sets the dropout rate. |
double LearningRate { get; set; } |
Gets or sets the learning rate. |
int FeedForwardDimension { get; set; } |
Gets or sets the feedforward network dimension. |
int ModelDimension { get; set; } |
Gets or sets the model dimension (embedding size). |
int NumFeatures { get; set; } |
Gets or sets the number of input features (variables/channels). |
int NumHeads { get; set; } |
Gets or sets the number of attention heads. |
int NumLayers { get; set; } |
Gets or sets the number of transformer encoder layers. |
int PredictionHorizon { get; set; } |
Gets or sets the prediction horizon. |
int SequenceLength { get; set; } |
Gets or sets the input sequence length. |
Methods
| Method | Summary |
|---|---|
List<string> Validate() |
Validates the options and returns any validation errors. |