FEDformer<T>
Models & Types · AiDotNet.Finance.Forecasting.Transformers
FEDformer (Frequency Enhanced Decomposed Transformer) for long-term time series forecasting.
Inheritance: NeuralNetworkBase<T> → FinancialModelBase<T> → ForecastingModelBase<T> → FEDformer<T>
For Beginners
FEDformer is like analyzing music by frequencies instead of individual notes. Standard transformers look at every time step (expensive), but FEDformer converts to frequency domain where patterns are simpler and operations are faster.
Key innovations:
- Frequency Attention: Uses Fourier/Wavelet transforms for O(N) complexity
- Decomposition: Separates trend (overall direction) from seasonal (repeating patterns)
- Random Mode Selection: Keeps only important frequencies for efficiency
How It Works
FEDformer achieves linear complexity O(N) by performing attention in the frequency domain using Fourier or Wavelet transforms. It also decomposes time series into trend and seasonal components for better interpretability and forecasting accuracy.
Reference: Zhou et al., “FEDformer: Frequency Enhanced Decomposed Transformer for Long-term Series Forecasting”,
ICML 2022. https://arxiv.org/abs/2201.12740
Example
// Create a FEDformer for long-term forecasting with frequency-domain attention
// Achieves O(N) complexity via Fourier/Wavelet transforms instead of O(N^2) attention
var architecture = new NeuralNetworkArchitecture<double>(
inputType: InputType.OneDimensional,
taskType: NeuralNetworkTaskType.Regression,
inputHeight: 96, inputWidth: 7, inputDepth: 1, outputSize: 24);
// Training mode with frequency-enhanced decomposed attention
var model = new FEDformer<double>(architecture);
// ONNX inference mode with pre-trained model
var onnxModel = new FEDformer<double>(architecture, "fedformer_etth2.onnx");
Constructors
| Constructor | Summary |
|---|---|
FEDformer(NeuralNetworkArchitecture<T> architecture, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer, ILossFunction<T> lossFunction, FEDformerOptions<T> options) |
Creates a FEDformer network using native library layers. |
FEDformer(NeuralNetworkArchitecture<T> architecture, string onnxModelPath, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer, ILossFunction<T> lossFunction, FEDformerOptions<T> options) |
Creates a FEDformer network using a pretrained ONNX model. |
Properties
| Property | Summary |
|---|---|
bool IsChannelIndependent { get; } |
|
bool SupportsTraining { get; } |
|
bool UseNativeMode { get; } |
|
int NumFeatures { get; } |
|
int PatchSize { get; } |
|
int PredictionHorizon { get; } |
|
int SequenceLength { get; } |
|
int Stride { get; } |
Methods
| Method | Summary |
|---|---|
Dictionary<string, T> Evaluate(Tensor<T> inputs, Tensor<T> targets) |
|
Dictionary<string, T> GetFinancialMetrics() |
|
IReadOnlyCollection<string> LoadReferenceWeights(Func<string, int[], double[]> read) |
Loads the official FEDformer’s state dict straight from source into the weights. |
ModelMetadata<T> GetModelMetadata() |
|
Tensor<T> ApplyInstanceNormalization(Tensor<T> input) |
|
Tensor<T> AutoregressiveForecast(Tensor<T> input, int steps) |