Crossformer<T>

Models & Types · AiDotNet.Finance.Forecasting.Transformers

Crossformer (Cross-Dimension Transformer) neural network for multivariate time series forecasting.

Inheritance: NeuralNetworkBase<T> → FinancialModelBase<T> → ForecastingModelBase<T> → Crossformer<T>

For Beginners

Crossformer is designed for data where:

  • Multiple variables influence each other (like stocks in the same sector)
  • Both time patterns AND variable relationships matter
  • You need to capture how one variable’s past affects another’s future

Key innovations:

  • Two-Stage Attention (TSA): Alternates between cross-time and cross-dimension attention
  • Dimension-Segment Embedding: Embeds each variable-segment pair
  • Hierarchical Structure: Processes at multiple time scales

How It Works

Crossformer uses a two-stage attention mechanism that captures both temporal dependencies and cross-variable relationships simultaneously. It’s particularly effective for multivariate time series where different variables interact in complex ways.

Reference: Zhang et al., “Crossformer: Transformer Utilizing Cross-Dimension Dependency for Multivariate Time Series Forecasting”, ICLR 2023. https://openreview.net/forum?id=vSVLM2j9eie

Example

// Create a Crossformer for multivariate forecasting with cross-dimension attention
// Captures both temporal and cross-variable dependencies simultaneously
var architecture = new NeuralNetworkArchitecture<double>(
    inputType: InputType.OneDimensional,
    taskType: NeuralNetworkTaskType.Regression,
    inputHeight: 96, inputWidth: 7, inputDepth: 1, outputSize: 24);
            
// Training mode with two-stage attention (cross-time + cross-dimension)
var model = new Crossformer<double>(architecture);
            
// ONNX inference mode with pre-trained model
var onnxModel = new Crossformer<double>(architecture, "crossformer_weather.onnx");

Constructors

Constructor Summary
Crossformer(NeuralNetworkArchitecture<T> architecture, CrossformerOptions<T> options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer, ILossFunction<T> lossFunction) Creates a Crossformer network in native mode for training from scratch.
Crossformer(NeuralNetworkArchitecture<T> architecture, string onnxModelPath, CrossformerOptions<T> options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer, ILossFunction<T> lossFunction) Creates a Crossformer network using pretrained ONNX model.

Properties

Property Summary
bool IsChannelIndependent { get; }
bool SupportsTraining { get; }
bool UseNativeMode { get; }
int NumFeatures { get; }
int PatchSize { get; }
int PredictionHorizon { get; }
int SequenceLength { get; }
int Stride { get; }

Methods

Method Summary
Dictionary<string, T> Evaluate(Tensor<T> predictions, Tensor<T> actuals)
Dictionary<string, T> GetFinancialMetrics()
ModelMetadata<T> GetModelMetadata()
Tensor<T> ApplyInstanceNormalization(Tensor<T> input)
Tensor<T> AutoregressiveForecast(Tensor<T> input, int steps)