ScoreGrad<T>

Models & Types · AiDotNet.Finance.Probabilistic

ScoreGrad (Score-based Gradient Model) for probabilistic time series forecasting using score matching.

Inheritance: NeuralNetworkBase<T> → FinancialModelBase<T> → ForecastingModelBase<T> → ScoreGrad<T>

For Beginners

ScoreGrad uses a principled probabilistic approach called “score matching”:

The Key Insight: Instead of directly generating samples or predicting noise, ScoreGrad learns the “score function” - the direction in which probability increases. By following this direction (with some randomness), the model can generate realistic time series from noise.

What is the Score Function? The score is ∇_x log p(x) - the gradient of log probability with respect to the data x.

  • If you’re at point x, the score tells you which direction has higher probability
  • Following the score uphill leads to likely data points
  • It’s like having a compass that points toward “good” data

How ScoreGrad Works:

  1. Score Network: Train a neural network to predict the score at different noise levels
  2. Denoising Score Matching: Learn scores by adding noise and learning to denoise
  3. Langevin Dynamics: Sample by iteratively following the score plus random noise
  4. Annealing: Start with high noise (easy to sample), gradually reduce for refinement

ScoreGrad Architecture:

  • Score Network: Predicts ∇_x log p(x|σ) conditioned on noise level σ
  • Noise Embedding: Sinusoidal encoding of current noise level
  • Residual Blocks: Deep architecture for learning complex score functions
  • Output: Gradient direction at each position in the forecast

Key Benefits:

  • Principled probabilistic foundation
  • Flexible sampling (adjust step size and iterations)
  • Natural uncertainty quantification
  • Works well for complex multivariate dynamics

How It Works

ScoreGrad is a score-based generative model that learns the gradient of the log probability density function (score) and uses it for sampling via Langevin dynamics.

Reference: Yan et al., “ScoreGrad: Multivariate Probabilistic Time Series Forecasting with Continuous Energy-based Generative Models”, 2021. https://arxiv.org/abs/2106.10121

Example

// Create a ScoreGrad model for energy-based time series forecasting
var architecture = new NeuralNetworkArchitecture<double>(
    inputType: InputType.OneDimensional,
    taskType: NeuralNetworkTaskType.Regression,
    inputHeight: 100, inputWidth: 1, inputDepth: 1, outputSize: 24);
var model = new ScoreGrad<double>(architecture);
            
// Or load a pre-trained ONNX model for score-based forecasting
var onnxModel = new ScoreGrad<double>(architecture, "scoregrad.onnx");

Constructors

Constructor Summary
ScoreGrad(NeuralNetworkArchitecture<T> architecture, ScoreGradOptions<T> options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer, ILossFunction<T> lossFunction) Initializes a new instance of the ScoreGrad model in native mode for training.
ScoreGrad(NeuralNetworkArchitecture<T> architecture, string onnxModelPath, ScoreGradOptions<T> options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer, ILossFunction<T> lossFunction) Initializes a new instance of the ScoreGrad model in ONNX mode for inference.

Properties

Property Summary
bool IsChannelIndependent { get; }
bool SupportsTraining { get; } Gets whether the model supports training (native mode only).
bool UseNativeMode { get; }
int ContextLength { get; } Gets the context length (input history length).
int ForecastHorizon { get; } Gets the forecast horizon.
int NumFeatures { get; }
int NumLangevinSteps { get; } Gets the number of Langevin steps per noise level.
int NumNoiseScales { get; } Gets the number of noise scales for multi-scale score matching.
int NumSamples { get; } Gets the number of samples for uncertainty estimation.
int PatchSize { get; }
int PredictionHorizon { get; }
int SequenceLength { get; }
int Stride { get; }

Methods

Method Summary
Dictionary<string, T> Evaluate(Tensor<T> predictions, Tensor<T> actuals) Evaluates forecast quality against actual values.
Dictionary<string, T> GetFinancialMetrics() Gets financial-specific metrics.
ModelMetadata<T> GetModelMetadata() Gets metadata about the ScoreGrad model.
Tensor<T> ApplyInstanceNormalization(Tensor<T> input) Applies instance normalization (identity for ScoreGrad).
Tensor<T> AutoregressiveForecast(Tensor<T> input, int steps) Performs autoregressive forecasting.
Tensor<T> Forward(Tensor<T> input) Performs the forward pass through all layers.
Tensor<T> ForwardForTraining(Tensor<T> input) Trains the ScoreGrad model using denoising score matching.
ValueTuple<Tensor<T>, Tensor<T>, Tensor<T>> ForecastWithIntervals(Tensor<T> input, double confidenceLevel) Generates forecasts with prediction intervals.