Crossformer<T>
Models & Types · AiDotNet.Finance.Forecasting.Transformers
Crossformer (Cross-Dimension Transformer) neural network for multivariate time series forecasting.
Inheritance: NeuralNetworkBase<T> → FinancialModelBase<T> → ForecastingModelBase<T> → Crossformer<T>
For Beginners
Crossformer is designed for data where:
- Multiple variables influence each other (like stocks in the same sector)
- Both time patterns AND variable relationships matter
- You need to capture how one variable’s past affects another’s future
Key innovations:
- Two-Stage Attention (TSA): Alternates between cross-time and cross-dimension attention
- Dimension-Segment Embedding: Embeds each variable-segment pair
- Hierarchical Structure: Processes at multiple time scales
How It Works
Crossformer uses a two-stage attention mechanism that captures both temporal dependencies and cross-variable relationships simultaneously. It’s particularly effective for multivariate time series where different variables interact in complex ways.
Reference: Zhang et al., “Crossformer: Transformer Utilizing Cross-Dimension
Dependency for Multivariate Time Series Forecasting”, ICLR 2023.
https://openreview.net/forum?id=vSVLM2j9eie
Example
// Create a Crossformer for multivariate forecasting with cross-dimension attention
// Captures both temporal and cross-variable dependencies simultaneously
var architecture = new NeuralNetworkArchitecture<double>(
inputType: InputType.OneDimensional,
taskType: NeuralNetworkTaskType.Regression,
inputHeight: 96, inputWidth: 7, inputDepth: 1, outputSize: 24);
// Training mode with two-stage attention (cross-time + cross-dimension)
var model = new Crossformer<double>(architecture);
// ONNX inference mode with pre-trained model
var onnxModel = new Crossformer<double>(architecture, "crossformer_weather.onnx");
Constructors
| Constructor | Summary |
|---|---|
Crossformer(NeuralNetworkArchitecture<T> architecture, CrossformerOptions<T> options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer, ILossFunction<T> lossFunction) |
Creates a Crossformer network in native mode for training from scratch. |
Crossformer(NeuralNetworkArchitecture<T> architecture, string onnxModelPath, CrossformerOptions<T> options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer, ILossFunction<T> lossFunction) |
Creates a Crossformer network using pretrained ONNX model. |
Properties
| Property | Summary |
|---|---|
bool IsChannelIndependent { get; } |
|
bool SupportsTraining { get; } |
|
bool UseNativeMode { get; } |
|
int NumFeatures { get; } |
|
int PatchSize { get; } |
|
int PredictionHorizon { get; } |
|
int SequenceLength { get; } |
|
int Stride { get; } |
Methods
| Method | Summary |
|---|---|
Dictionary<string, T> Evaluate(Tensor<T> predictions, Tensor<T> actuals) |
|
Dictionary<string, T> GetFinancialMetrics() |
|
ModelMetadata<T> GetModelMetadata() |
|
Tensor<T> ApplyInstanceNormalization(Tensor<T> input) |
|
Tensor<T> AutoregressiveForecast(Tensor<T> input, int steps) |