SimMTM<T>

Models & Types · AiDotNet.Finance.Forecasting.Foundation

SimMTM — Simple Pre-Training Framework for Masked Time-Series Modeling.

Inheritance: NeuralNetworkBase<T> → FinancialModelBase<T> → ForecastingModelBase<T> → TimeSeriesFoundationModelBase<T> → SimMTM<T>

For Beginners

SimMTM learns about time series by playing a fill-in-the-blank game. Parts of the data are hidden (masked), and the model must reconstruct them. What makes it special is that it looks at similar series in the training batch for clues, like asking a friend who has seen similar patterns. This pre-training approach helps the model learn robust representations that transfer well to forecasting tasks.

How It Works

SimMTM combines masked time series modeling with series-level similarity learning, recovering masked series by aggregating from similar unmasked series in the batch. It uses a patch-based transformer with a similarity-weighted reconstruction objective.

Reference: Dong et al., “SimMTM: A Simple Pre-Training Framework for Masked Time-Series Modeling”, NeurIPS 2023.

Example

// Create a SimMTM model for masked time series pre-training with similarity learning
// Reconstructs masked series using similar unmasked series from the training batch
var architecture = new NeuralNetworkArchitecture<double>(
    inputType: InputType.OneDimensional,
    taskType: NeuralNetworkTaskType.Regression,
    inputHeight: 512, inputWidth: 1, inputDepth: 1, outputSize: 24);
            
// Training mode with similarity-weighted masked reconstruction
var model = new SimMTM<double>(architecture);
            
// ONNX inference mode with pre-trained model
var onnxModel = new SimMTM<double>(architecture, "simmtm.onnx");

Constructors

Constructor Summary
SimMTM(NeuralNetworkArchitecture<T> architecture, SimMTMOptions<T> options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer, ILossFunction<T> lossFunction) Creates a SimMTM model in native mode for training or fine-tuning.
SimMTM(NeuralNetworkArchitecture<T> architecture, string onnxModelPath, SimMTMOptions<T> options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer, ILossFunction<T> lossFunction) Creates a SimMTM model using a pretrained ONNX model.

Properties

Property Summary
FoundationModelSize ModelSize { get; }
bool IsChannelIndependent { get; }
bool SupportsTraining { get; }
bool UseNativeMode { get; }
int MaxContextLength { get; }
int MaxPredictionHorizon { get; }
int NumFeatures { get; }
int PatchSize { get; }
int PredictionHorizon { get; }
int SequenceLength { get; }
int Stride { get; }

Methods

Method Summary
Dictionary<string, T> Evaluate(Tensor<T> predictions, Tensor<T> actuals)
Dictionary<string, T> GetFinancialMetrics()
Dictionary<string, Tensor<T>> GetNamedLayerActivations(Tensor<T> input) Per-layer activations for the introspection tests.
ModelMetadata<T> GetModelMetadata()
Tensor<T> ApplyInstanceNormalization(Tensor<T> input)
Tensor<T> AutoregressiveForecast(Tensor<T> input, int steps)
Tensor<T> ForwardForTraining(Tensor<T> input) Training forward.
ValueTuple<Tensor<T>, Tensor<T>> PretrainSimilarityWeightedReconstruction(Tensor<T> input, int? seed) Performs one SimMTM similarity-weighted masked-reconstruction pretraining forward pass per Dong et al.