VisionTS<T>

Models & Types · AiDotNet.Finance.Forecasting.Foundation

VisionTS — Visual Masked Autoencoders as Zero-Shot Time Series Forecasters.

Inheritance: NeuralNetworkBase<T> → FinancialModelBase<T> → ForecastingModelBase<T> → TimeSeriesFoundationModelBase<T> → VisionTS<T>

For Beginners

VisionTS takes a surprising approach: it converts time series data into images and uses a vision model (originally trained on photos) to forecast future values. The data is arranged in a 2D grid like pixels, and the vision model fills in the missing parts, effectively predicting future values. This works because patterns in time series grids resemble visual textures that vision models already understand.

How It Works

VisionTS repurposes Visual Masked Autoencoders (MAE) pretrained on images for time series forecasting. It converts time series into 2D image-like patch grids, processes them with a pretrained ViT encoder, and reconstructs/forecasts using the decoder. This cross-modal transfer demonstrates that vision foundation models generalize to time series.

Reference: “VisionTS: Visual Masked Autoencoders as Zero-Shot Time Series Forecasters”, ICML 2025.

Example

// Create a VisionTS model that repurposes visual MAE for time series forecasting
// Converts time series into 2D image-like patch grids for cross-modal transfer
var architecture = new NeuralNetworkArchitecture<double>(
    inputType: InputType.OneDimensional,
    taskType: NeuralNetworkTaskType.Regression,
    inputHeight: 512, inputWidth: 1, inputDepth: 1, outputSize: 24);
            
// Training mode with ViT encoder and MAE decoder
var model = new VisionTS<double>(architecture);
            
// ONNX inference mode with pre-trained vision model
var onnxModel = new VisionTS<double>(architecture, "visionts.onnx");

Constructors

Constructor Summary
VisionTS(NeuralNetworkArchitecture<T> architecture, VisionTSOptions<T> options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer, ILossFunction<T> lossFunction) Creates a VisionTS model in native mode.
VisionTS(NeuralNetworkArchitecture<T> architecture, string onnxModelPath, VisionTSOptions<T> options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer, ILossFunction<T> lossFunction) Creates a VisionTS model using a pretrained ONNX model.

Properties

Property Summary
FoundationModelSize ModelSize { get; }
bool IsChannelIndependent { get; }
bool SupportsTraining { get; }
bool UseNativeMode { get; }
int MaxContextLength { get; }
int MaxPredictionHorizon { get; }
int NumFeatures { get; }
int PatchSize { get; }
int PredictionHorizon { get; }
int SequenceLength { get; }
int Stride { get; }

Methods

Method Summary
Dictionary<string, T> Evaluate(Tensor<T> predictions, Tensor<T> actuals)
Dictionary<string, T> GetFinancialMetrics()
Dictionary<string, Tensor<T>> GetNamedLayerActivations(Tensor<T> input)
ModelMetadata<T> GetModelMetadata()
Tensor<T> ApplyInstanceNormalization(Tensor<T> input)
Tensor<T> AutoregressiveForecast(Tensor<T> input, int steps)