IVideoLanguageModel<T>
Interfaces · AiDotNet.VisionLanguage.Interfaces
Interface for video-language models that process video frames for temporal understanding and QA.
Implements: IGenerativeVisionLanguageModel<T>
Known implementations: LLaVANeXTVideo<T>, LLaVAVideo<T>, LongVILA<T>, PLLaVA<T>, SlowFastLLaVA<T>, VideoChat2<T>, VideoLLaMA2<T>, VideoLLaMA3<T>, VideoLLaVA<T>
How It Works
Video-language models extend image-based VLMs to handle temporal sequences of video frames. Architectures include frame averaging, spatial-temporal convolution, slow/fast pathways, and long-context approaches for processing hour+ videos.
Properties
| Property | Summary |
|---|---|
int MaxFrames { get; } |
Gets the maximum number of video frames the model can process. |
string LanguageModelName { get; } |
Gets the name of the language model backbone. |
Methods
| Method | Summary |
|---|---|
Tensor<T> GenerateFromVideo(IReadOnlyList<Tensor<T>> frames, string prompt) |
Generates output from a sequence of video frames, optionally conditioned on a text prompt. |