FEDformer<T>

Models & Types · AiDotNet.Finance.Forecasting.Transformers

FEDformer (Frequency Enhanced Decomposed Transformer) for long-term time series forecasting.

Inheritance: NeuralNetworkBase<T> → FinancialModelBase<T> → ForecastingModelBase<T> → FEDformer<T>

For Beginners

FEDformer is like analyzing music by frequencies instead of individual notes. Standard transformers look at every time step (expensive), but FEDformer converts to frequency domain where patterns are simpler and operations are faster.

Key innovations:

  • Frequency Attention: Uses Fourier/Wavelet transforms for O(N) complexity
  • Decomposition: Separates trend (overall direction) from seasonal (repeating patterns)
  • Random Mode Selection: Keeps only important frequencies for efficiency

How It Works

FEDformer achieves linear complexity O(N) by performing attention in the frequency domain using Fourier or Wavelet transforms. It also decomposes time series into trend and seasonal components for better interpretability and forecasting accuracy.

Reference: Zhou et al., “FEDformer: Frequency Enhanced Decomposed Transformer for Long-term Series Forecasting”, ICML 2022. https://arxiv.org/abs/2201.12740

Example

// Create a FEDformer for long-term forecasting with frequency-domain attention
// Achieves O(N) complexity via Fourier/Wavelet transforms instead of O(N^2) attention
var architecture = new NeuralNetworkArchitecture<double>(
    inputType: InputType.OneDimensional,
    taskType: NeuralNetworkTaskType.Regression,
    inputHeight: 96, inputWidth: 7, inputDepth: 1, outputSize: 24);
            
// Training mode with frequency-enhanced decomposed attention
var model = new FEDformer<double>(architecture);
            
// ONNX inference mode with pre-trained model
var onnxModel = new FEDformer<double>(architecture, "fedformer_etth2.onnx");

Constructors

Constructor Summary
FEDformer(NeuralNetworkArchitecture<T> architecture, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer, ILossFunction<T> lossFunction, FEDformerOptions<T> options) Creates a FEDformer network using native library layers.
FEDformer(NeuralNetworkArchitecture<T> architecture, string onnxModelPath, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer, ILossFunction<T> lossFunction, FEDformerOptions<T> options) Creates a FEDformer network using a pretrained ONNX model.

Properties

Property Summary
bool IsChannelIndependent { get; }
bool SupportsTraining { get; }
bool UseNativeMode { get; }
int NumFeatures { get; }
int PatchSize { get; }
int PredictionHorizon { get; }
int SequenceLength { get; }
int Stride { get; }

Methods

Method Summary
Dictionary<string, T> Evaluate(Tensor<T> inputs, Tensor<T> targets)
Dictionary<string, T> GetFinancialMetrics()
IReadOnlyCollection<string> LoadReferenceWeights(Func<string, int[], double[]> read) Loads the official FEDformer’s state dict straight from source into the weights.
ModelMetadata<T> GetModelMetadata()
Tensor<T> ApplyInstanceNormalization(Tensor<T> input)
Tensor<T> AutoregressiveForecast(Tensor<T> input, int steps)