MOMENT<T>
Models & Types · AiDotNet.Finance.Forecasting.Foundation
MOMENT (Multi-task Optimization through Masked Encoding for Time series) foundation model.
Inheritance: NeuralNetworkBase<T> → FinancialModelBase<T> → ForecastingModelBase<T> → TimeSeriesFoundationModelBase<T> → MOMENT<T>
For Beginners
MOMENT is the first true multi-task time series foundation model:
Architecture Overview:
- Patch Embedding: Divides the input into fixed-length patches (e.g., 64 steps each)
- RevIN: Reversible Instance Normalization handles different scales automatically
- T5 Encoder: A stack of transformer encoder layers processes the patches
- Task Heads: Separate output heads for each supported task
Multi-Task Capability:
- Forecasting: Linear projection from encoder output to future values
- Anomaly Detection: Reconstruction-based — anomalies have high reconstruction error
- Classification: Pooled encoder output fed to a classification head
- Imputation: Masked patches reconstructed using unmasked context
- Embedding: Mean-pooled encoder output serves as the representation
Key Insight: MOMENT’s pretraining uses masked reconstruction (like BERT), which naturally supports all five tasks without task-specific pretraining.
Model Sizes (MOMENT family):
- MOMENT-Small: ~40M parameters
- MOMENT-Base: ~385M parameters (default)
- MOMENT-Large: ~1B+ parameters
How It Works
MOMENT is a family of open time series foundation models from Carnegie Mellon University. It uses a T5-based encoder-only transformer with patch-based input and RevIN to handle five downstream tasks: forecasting, anomaly detection, classification, imputation, and embedding generation — all from a single pretrained backbone.
Reference: Goswami et al., “MOMENT: A Family of Open Time-Series Foundation Models”,
ICML 2024. https://arxiv.org/abs/2402.03885
Thread Safety: This class is NOT thread-safe. Create separate instances for concurrent usage.
Example
// Create a MOMENT multi-task time series foundation model
// Supports forecasting, anomaly detection, classification, imputation, and embedding
var architecture = new NeuralNetworkArchitecture<double>(
inputType: InputType.OneDimensional,
taskType: NeuralNetworkTaskType.Regression,
inputHeight: 512, inputWidth: 1, inputDepth: 1, outputSize: 24);
// Training mode with T5-based encoder and task-specific heads
var model = new MOMENT<double>(architecture);
// ONNX inference mode with pre-trained model
var onnxModel = new MOMENT<double>(architecture, "moment_base.onnx");
Constructors
| Constructor | Summary |
|---|---|
MOMENT(NeuralNetworkArchitecture<T> architecture, MOMENTOptions<T> options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer, ILossFunction<T> lossFunction) |
Creates a MOMENT model in native mode for training or fine-tuning. |
MOMENT(NeuralNetworkArchitecture<T> architecture, string onnxModelPath, MOMENTOptions<T> options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer, ILossFunction<T> lossFunction) |
Creates a MOMENT model using a pretrained ONNX model. |
Properties
| Property | Summary |
|---|---|
FoundationModelSize ModelSize { get; } |
|
IReadOnlyList<TimeSeriesFoundationModelTask> SupportedTasks { get; } |
|
TimeSeriesFoundationModelTask CurrentTask { get; } |
|
bool IsChannelIndependent { get; } |
|
bool SupportsTraining { get; } |
|
bool UseNativeMode { get; } |
|
int MaxContextLength { get; } |
|
int MaxPredictionHorizon { get; } |
|
int NumFeatures { get; } |
|
int PatchSize { get; } |
|
int PredictionHorizon { get; } |
|
int SequenceLength { get; } |
|
int Stride { get; } |
Methods
| Method | Summary |
|---|---|
Dictionary<string, T> Evaluate(Tensor<T> predictions, Tensor<T> actuals) |
|
Dictionary<string, T> GetFinancialMetrics() |
|
Dictionary<string, Tensor<T>> GetNamedLayerActivations(Tensor<T> input) |
Captures per-layer activations of the native forward pass. |
ModelMetadata<T> GetModelMetadata() |
|
Tensor<T> ApplyInstanceNormalization(Tensor<T> input) |
|
Tensor<T> AutoregressiveForecast(Tensor<T> input, int steps) |
|
Tensor<T> Classify(Tensor<T> series, int numClasses) |
|
Tensor<T> DetectAnomalies(Tensor<T> series, double? threshold) |
|
Tensor<T> Embed(Tensor<T> series) |
|
Tensor<T> Impute(Tensor<T> series, Tensor<T> mask) |