SlowFastLLaVA<T>

Models & Types · AiDotNet.VisionLanguage.VideoLanguage

SlowFast-LLaVA: token-efficient slow/fast pathways for long video.

Inheritance: NeuralNetworkBase<T> → VisionLanguageModelBase<T> → SlowFastLLaVA<T>

Implements: IVideoLanguageModel<T>

For Beginners

SlowFast-LLaVA is a training-free video-language model using dual pathways for efficient long video understanding. Default values follow the original paper settings.

How It Works

SlowFast-LLaVA (Meta, 2025) is a training-free video understanding baseline that uses dual slow/fast pathways for token-efficient long video processing. The slow pathway processes fewer frames at high spatial resolution, while the fast pathway samples many frames at low spatial resolution, combining detailed spatial and broad temporal information efficiently.

References:

  • Paper: “SlowFast-LLaVA: A Strong Training-Free Baseline for Video Large Language Models” (Meta, 2025)

Example

// Create a SlowFast-LLaVA model for long video understanding
// with token-efficient dual slow/fast pathways for temporal modeling
var architecture = new NeuralNetworkArchitecture<double>(
    inputType: InputType.TwoDimensional,
    taskType: NeuralNetworkTaskType.ImageClassification,
    inputHeight: 224, inputWidth: 224, inputDepth: 3, outputSize: 512);
            
// ONNX inference mode with pre-trained model
var model = new SlowFastLLaVA<double>(architecture, "slowfastllava.onnx");
            
// Training mode with native layers
var trainModel = new SlowFastLLaVA<double>(architecture, new SlowFastLLaVAOptions());

Constructors

Constructor Summary
SlowFastLLaVA(NeuralNetworkArchitecture<T> architecture, SlowFastLLaVAOptions options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer)
SlowFastLLaVA(NeuralNetworkArchitecture<T> architecture, string modelPath, SlowFastLLaVAOptions options)

Properties

Property Summary
int DecoderEmbeddingDim { get; }
int EmbeddingDimension { get; }
int MaxFrames { get; }
int MaxGenerationLength { get; }
string LanguageModelName { get; }

Methods

Method Summary
ModelMetadata<T> GetModelMetadata()
Tensor<T> EncodeImage(Tensor<T> image)
Tensor<T> GenerateFromImage(Tensor<T> image, string prompt) Generates from a single image using SlowFast-LLaVA’s slow pathway at full resolution.
Tensor<T> GenerateFromVideo(IReadOnlyList<Tensor<T>> frames, string prompt) Generates output from video frames using SlowFast-LLaVA’s dual-pathway architecture.