LagLlama<T>
Models & Types · AiDotNet.Finance.Forecasting.Foundation
Lag-Llama foundation model for probabilistic time series forecasting.
Inheritance: NeuralNetworkBase<T> → FinancialModelBase<T> → ForecastingModelBase<T> → LagLlama<T>
For Beginners
Lag-Llama brings LLM innovations to time series forecasting:
The Lag Feature Innovation: Instead of just looking at recent values, Lag-Llama creates features from specific past points:
- Lag-1: Yesterday’s value (immediate trend)
- Lag-7: Same day last week (weekly pattern)
- Lag-365: Same day last year (annual pattern)
This lets the model explicitly see patterns at different time scales without needing a very long context window.
Llama Architecture Adaptations: Lag-Llama adopts key Llama innovations:
- RMSNorm: Simpler, faster layer normalization
- SwiGLU: Improved MLP activation function
- RoPE: Rotary Position Embeddings for better position encoding
- Causal Attention: Each position only sees earlier positions
Probabilistic Forecasting: Unlike models that output single values, Lag-Llama outputs distribution parameters:
- For Student-t: degrees of freedom (nu), location (mu), scale (sigma)
- Allows uncertainty quantification: “The forecast is 100 ± 15”
- Enables risk-aware decisions: “There’s a 5% chance it exceeds 130”
Why Student-t Distribution?
- Has heavier tails than Normal (better for extreme events)
- Degrades gracefully to Normal as nu → infinity
- More robust to outliers in training data
Zero-Shot Capability: Pre-trained on diverse time series, Lag-Llama can forecast new series without training.
How It Works
Lag-Llama adapts the Llama large language model architecture for time series forecasting. It uses lag-based features to capture temporal patterns at multiple scales and outputs probabilistic forecasts via distribution parameter prediction.
Reference: Rasul et al., “Lag-Llama: Towards Foundation Models for Probabilistic Time Series Forecasting”, 2024.
https://arxiv.org/abs/2310.08278
Example
// Create a Lag-Llama for probabilistic time series forecasting
// Adapts the Llama LLM architecture with lag-based features for multi-scale temporal patterns
var architecture = new NeuralNetworkArchitecture<double>(
inputType: InputType.OneDimensional,
taskType: NeuralNetworkTaskType.Regression,
inputHeight: 512, inputWidth: 1, inputDepth: 1, outputSize: 24);
// Training mode with RoPE, SwiGLU, and Student-t distribution output
var model = new LagLlama<double>(architecture);
// ONNX inference mode with pre-trained model
var onnxModel = new LagLlama<double>(architecture, "lag_llama.onnx");
Constructors
| Constructor | Summary |
|---|---|
LagLlama(NeuralNetworkArchitecture<T> architecture, LagLlamaOptions<T> options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer, ILossFunction<T> lossFunction) |
Creates a Lag-Llama model in native mode for training or fine-tuning. |
LagLlama(NeuralNetworkArchitecture<T> architecture, string onnxModelPath, LagLlamaOptions<T> options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer, ILossFunction<T> lossFunction) |
Creates a Lag-Llama model using pretrained ONNX model. |
Properties
| Property | Summary |
|---|---|
bool IsChannelIndependent { get; } |
|
bool SupportsTraining { get; } |
|
bool UseNativeMode { get; } |
|
int NumFeatures { get; } |
|
int PatchSize { get; } |
|
int PredictionHorizon { get; } |
|
int SequenceLength { get; } |
|
int Stride { get; } |
|
int[] LagIndices { get; } |
Gets the lag indices used for feature extraction. |
string DistributionOutput { get; } |
Gets the distribution type used for probabilistic output. |
Methods
| Method | Summary |
|---|---|
Dictionary<string, T> Evaluate(Tensor<T> predictions, Tensor<T> actuals) |
|
Dictionary<string, T> GetFinancialMetrics() |
|
Dictionary<string, Tensor<T>> GetNamedLayerActivations(Tensor<T> input) |
|
ModelMetadata<T> GetModelMetadata() |
|
Tensor<T> ApplyInstanceNormalization(Tensor<T> input) |
|
Tensor<T> AutoregressiveForecast(Tensor<T> input, int steps) |