TimeMAE<T>

Models & Types · AiDotNet.Finance.Forecasting.Foundation

TimeMAE — Masked Autoencoder for Time Series with Decoupled Masked Autoencoders.

Inheritance: NeuralNetworkBase<T> → FinancialModelBase<T> → ForecastingModelBase<T> → TimeSeriesFoundationModelBase<T> → TimeMAE<T>

For Beginners

TimeMAE learns about time series by hiding random chunks of data and training a model to fill them back in, similar to how a student learns vocabulary by doing fill-in-the-blank exercises. The model only processes the visible chunks with a large encoder and uses a smaller decoder to reconstruct the hidden ones, making training very efficient. The learned representations transfer well to forecasting tasks.

How It Works

TimeMAE applies masked autoencoding to time series, randomly masking patches of the input and training a transformer to reconstruct the missing patches, learning rich temporal representations. It uses an asymmetric encoder-decoder architecture where the encoder processes visible patches only.

Reference: Cheng et al., “TimeMAE: Self-Supervised Representations of Time Series with Decoupled Masked Autoencoders”, 2023.

Example

// Create a TimeMAE masked autoencoder for time series representation learning
// Asymmetric encoder-decoder learns by reconstructing randomly masked patches
var architecture = new NeuralNetworkArchitecture<double>(
    inputType: InputType.OneDimensional,
    taskType: NeuralNetworkTaskType.Regression,
    inputHeight: 512, inputWidth: 1, inputDepth: 1, outputSize: 24);
            
// Training mode with masked patch reconstruction
var model = new TimeMAE<double>(architecture);
            
// ONNX inference mode with pre-trained model
var onnxModel = new TimeMAE<double>(architecture, "timemae.onnx");

Constructors

Constructor Summary
TimeMAE(NeuralNetworkArchitecture<T> architecture, TimeMAEOptions<T> options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer, ILossFunction<T> lossFunction) Creates a TimeMAE model in native mode for training or fine-tuning.
TimeMAE(NeuralNetworkArchitecture<T> architecture, string onnxModelPath, TimeMAEOptions<T> options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer, ILossFunction<T> lossFunction) Creates a TimeMAE model using a pretrained ONNX model.

Properties

Property Summary
FoundationModelSize ModelSize { get; }
bool IsChannelIndependent { get; }
bool SupportsTraining { get; }
bool UseNativeMode { get; }
int MaxContextLength { get; }
int MaxPredictionHorizon { get; }
int NumFeatures { get; }
int PatchSize { get; }
int PredictionHorizon { get; }
int SequenceLength { get; }
int Stride { get; }

Methods

Method Summary
Dictionary<string, T> Evaluate(Tensor<T> predictions, Tensor<T> actuals)
Dictionary<string, T> GetFinancialMetrics()
Dictionary<string, Tensor<T>> GetNamedLayerActivations(Tensor<T> input)
ModelMetadata<T> GetModelMetadata()
Tensor<T> ApplyInstanceNormalization(Tensor<T> input)
Tensor<T> AutoregressiveForecast(Tensor<T> input, int steps)
ValueTuple<Tensor<T>, Tensor<T>> PretrainMaskedReconstruction(Tensor<T> input, int? seed) Performs one TimeMAE masked-reconstruction pretraining forward pass per Cheng et al.