TemporalFusionTransformer<T>
Models & Types · AiDotNet.TimeSeries
Implements the Temporal Fusion Transformer (TFT) per Lim et al. (2021). Architecture retained here: input embedding → variable selection (softmax-weighted feature combination) → static-enrichment GRN → interpretable multi-head attention → post-attention gated skip (GRN) → forecast head.
Inheritance: TimeSeriesModelBase<T> → TemporalFusionTransformer<T>
Implements: IGeneratedParameterRegistrar<T>
How It Works
Tape-based training. The entire forward pass (ForwardBatch) is
expressed with batched Engine.Tensor* ops, so a
GradientTape produces the whole backward pass
automatically (no hand-derived gradients) and every op is GPU-dispatchable. Windows of
the z-normalized series are stacked into a [B, L] batch and the full H-step horizon
is supervised per sample with the pinball / quantile loss summed over every configured
QuantileLevels level, so the model learns a
genuine predictive spread (see PredictQuantiles); the point forecast is the
median-level head.
Simplifications vs the full paper (documented per the tape-conversion campaign): (1) The sequential LSTM encoder-decoder is replaced by additive sinusoidal positional encodings feeding the self-attention block (a recurrent scan differentiates poorly on the tape and does not batch; positional encodings + attention recover temporal order). (2) For univariate series (the only exogenous inputs available here) variable selection reduces to a learned per-channel softmax gate over the embedded features rather than selection across multiple exogenous variables. The GRN, variable-selection gating, and interpretable multi-head attention — TFT’s defining components — are all retained and fully differentiable.
Example
using AiDotNet;
using AiDotNet.Data.Loaders;
using AiDotNet.TimeSeries;
using AiDotNet.Tensors.LinearAlgebra;
double[] series =
{
120, 135, 148, 160, 155, 170, 180, 195, 210, 198, 220, 235,
140, 155, 165, 178, 172, 190, 200, 215, 230, 218, 245, 260
};
var x = new Matrix<double>(series.Length, 1);
for (int i = 0; i < series.Length; i++) x[i, 0] = i;
var result = await new AiModelBuilder<double, Matrix<double>, Vector<double>>()
.ConfigureModel(new TemporalFusionTransformer<double>())
.ConfigureDataLoader(DataLoaders.FromMatrixVector(x, new Vector<double>(series)))
.BuildAsync();
var forecast = result.Predict(new Matrix<double>(6, 1));
Console.WriteLine($"TemporalFusionTransformer: forecast {forecast.Length} steps.");
Constructors
| Constructor | Summary |
|---|---|
TemporalFusionTransformer(TemporalFusionTransformerOptions<T> options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer) |
Methods
| Method | Summary |
|---|---|
ModelMetadata<T> GetModelMetadata() |
|
T PredictSingle(Vector<T> input) |
|
Vector<T> ForecastHorizon(Vector<T> input) |
Multi-horizon forecast for a single lookback window (H denormalized future values). |
Vector<T> Predict(Matrix<T> input) |
|
Vector<T> PredictQuantiles(Vector<T> input) |
Returns per-quantile forecasts as a flat vector laid out quantile-major: [q0_h0..q0_h(H-1), q1_h0..q1_h(H-1), ...] (index = qi*ForecastHorizon + h), in the same order as QuantileLevels. |