Timer<T>

Models & Types · AiDotNet.Finance.Forecasting.Foundation

Timer (Generative Pre-Training for Time Series) implementation.

Inheritance: NeuralNetworkBase<T> → FinancialModelBase<T> → ForecastingModelBase<T> → TimeSeriesFoundationModelBase<T> → Timer<T>

For Beginners

Timer brings GPT-style pre-training to time series:

The Key Insight: Just like GPT learns language by predicting the next token, Timer learns time series patterns by predicting future values. Pre-training on diverse datasets enables strong zero-shot transfer.

How It Works:

  1. Autoregressive Pre-training: Learn to predict future from past
  2. Masked Modeling: Learn to reconstruct masked portions
  3. Multi-scale Processing: Handle different temporal granularities
  4. Fine-tuning: Adapt to specific domains with minimal data

Advantages:

  • Strong zero-shot and few-shot performance
  • Generalizes across domains and frequencies
  • Efficient fine-tuning with minimal labeled data
  • Handles variable sequence lengths

How It Works

Timer is a generative pre-training approach for time series that uses autoregressive generation to learn rich temporal representations from diverse time series datasets, similar to GPT for language.

Timer-XL Enhancements (2024): Timer-XL extends the original Timer with long-context support and a unified framework for multiple forecasting tasks. Key improvements:

  • Extended context length (up to 4096 time steps)
  • Unified multi-task forecasting framework
  • Improved long-horizon performance

Reference: Liu et al., “Timer: Generative Pre-Training of Time Series”, 2024. https://arxiv.org/abs/2402.02368 Timer-XL: “Timer-XL: Long-Context Transformers for Unified Time Series Forecasting”, 2024.

Example

// Create a Timer model for generative pre-training on time series
// Uses autoregressive generation like GPT to learn rich temporal representations
var architecture = new NeuralNetworkArchitecture<double>(
    inputType: InputType.OneDimensional,
    taskType: NeuralNetworkTaskType.Regression,
    inputHeight: 512, inputWidth: 1, inputDepth: 1, outputSize: 24);
            
// Training mode with autoregressive and masked modeling objectives
var model = new Timer<double>(architecture);
            
// ONNX inference mode with pre-trained model
var onnxModel = new Timer<double>(architecture, "timer_base.onnx");

Constructors

Constructor Summary
Timer(NeuralNetworkArchitecture<T> architecture, TimerOptions<T> options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer, ILossFunction<T> lossFunction) Initializes a new instance in native mode for training.
Timer(NeuralNetworkArchitecture<T> architecture, string onnxModelPath, TimerOptions<T> options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer, ILossFunction<T> lossFunction) Initializes a new instance using an ONNX pretrained model.

Properties

Property Summary
FoundationModelSize ModelSize { get; }
bool IsChannelIndependent { get; }
bool SupportsTraining { get; }
bool UseAutoregressiveDecoding { get; } Gets whether autoregressive decoding is used.
bool UseNativeMode { get; }
double GenerationTemperature { get; } Gets the generation temperature.
double MaskRatio { get; } Gets the mask ratio for masked modeling.
int MaxContextLength { get; }
int MaxPredictionHorizon { get; }
int NumFeatures { get; }
int PatchSize { get; }
int PredictionHorizon { get; }
int SequenceLength { get; }
int Stride { get; }

Methods

Method Summary
Dictionary<string, T> Evaluate(Tensor<T> predictions, Tensor<T> actuals)
Dictionary<string, T> GetFinancialMetrics()
Dictionary<string, Tensor<T>> GetNamedLayerActivations(Tensor<T> input) Performs the forward pass through the network.
ModelMetadata<T> GetModelMetadata()
Tensor<T> ApplyInstanceNormalization(Tensor<T> input)
Tensor<T> AutoregressiveForecast(Tensor<T> input, int steps)
Tensor<T> MaskedPretraining(Tensor<T> input, Tensor<T> mask) Performs masked modeling pre-training step.