InternVideo2<T>

Models & Types · AiDotNet.Video.Understanding

InternVideo2: Scaling Video Foundation Models for Multimodal Video Understanding.

Inheritance: NeuralNetworkBase<T> → InternVideo2<T>

For Beginners

InternVideo2 understands video content by analyzing frames and learning relationships between visual content and language. It can:

  • Classify videos (what’s happening?)
  • Find videos matching text descriptions
  • Answer questions about video content
  • Generate video captions

Example usage (native mode for training):

Example usage (ONNX mode for inference only):

How It Works

InternVideo2 is a state-of-the-art video understanding model that combines:

  • Video-text contrastive learning
  • Masked video modeling
  • Video-text generative learning

Reference: “InternVideo2: Scaling Video Foundation Models for Multimodal Video Understanding” https://arxiv.org/abs/2403.15377

Constructors

Constructor Summary
InternVideo2() Creates an InternVideo2 model using native layers for training and inference.
InternVideo2(NeuralNetworkArchitecture<T> architecture, InternVideo2Options options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer, ILossFunction<T> lossFunction)
InternVideo2(NeuralNetworkArchitecture<T> architecture, string onnxModelPath, InternVideo2Options options) Creates an InternVideo2 model using a pretrained ONNX model for inference.

Properties

Property Summary
bool SupportsTraining { get; } Gets whether training is supported (only in native mode).

Methods

Method Summary
ModelMetadata<T> GetModelMetadata()
T ComputeSimilarity(Tensor<T> videoEmbedding, Tensor<T> textEmbedding) Computes similarity between video and text embeddings.
Tensor<T> EncodeVideo(Tensor<T> videoFrames) Encodes video frames into an embedding vector.