CrossformerOptions<T>
Options & Configuration · AiDotNet.Models.Options
Configuration options for the Crossformer model.
Inheritance: ModelOptions → CrossformerOptions<T>
For Beginners
These options control how the Crossformer model behaves:
Key settings:
- SegmentLength: How long each time segment is for cross-time attention.
- NumLayers: How deep the transformer is. More layers = more capacity but slower.
- NumHeads: Attention heads. More heads can capture different patterns.
- RouterTopK: For mixture of experts, how many experts to use per token.
Crossformer excels at capturing both temporal patterns and cross-variable relationships that are common in multivariate financial time series.
How It Works
Crossformer uses a cross-dimension attention mechanism that captures both temporal and cross-variable dependencies simultaneously through a two-stage attention structure.
Reference: Zhang et al., “Crossformer: Transformer Utilizing Cross-Dimension
Dependency for Multivariate Time Series Forecasting”, ICLR 2023.
https://openreview.net/forum?id=vSVLM2j9eie
Constructors
| Constructor | Summary |
|---|---|
CrossformerOptions() |
Properties
| Property | Summary |
|---|---|
ILossFunction<T> LossFunction { get; set; } |
Gets or sets the loss function for training. |
bool UseInstanceNormalization { get; set; } |
Gets or sets whether to use instance normalization (RevIN). |
bool UsePreNorm { get; set; } |
Gets or sets whether to use pre-norm (LayerNorm before attention/FFN). |
double AttentionDropout { get; set; } |
Gets or sets the attention dropout rate. |
double Dropout { get; set; } |
Gets or sets the dropout rate. |
double LearningRate { get; set; } |
Gets or sets the learning rate. |
int FeedForwardDimension { get; set; } |
Gets or sets the feedforward network dimension. |
int ModelDimension { get; set; } |
Gets or sets the model dimension (embedding size). |
int NumFeatures { get; set; } |
Gets or sets the number of input features. |
int NumHeads { get; set; } |
Gets or sets the number of attention heads. |
int NumLayers { get; set; } |
Gets or sets the number of transformer encoder layers. |
int PredictionHorizon { get; set; } |
Gets or sets the prediction horizon. |
int RouterTopK { get; set; } |
Gets or sets the top-K value for router in cross-dimension attention. |
int SegmentLength { get; set; } |
Gets or sets the segment length for cross-time attention. |
int SequenceLength { get; set; } |
Gets or sets the input sequence length. |
Methods
| Method | Summary |
|---|---|
List<string> Validate() |
Validates the options and returns any validation errors. |
int CalculateNumSegments() |
Calculates the number of segments based on sequence length and segment length. |