InternVideo2<T>
Models & Types · AiDotNet.Video.Understanding
InternVideo2: Scaling Video Foundation Models for Multimodal Video Understanding.
Inheritance: NeuralNetworkBase<T> → InternVideo2<T>
For Beginners
InternVideo2 understands video content by analyzing frames and learning relationships between visual content and language. It can:
- Classify videos (what’s happening?)
- Find videos matching text descriptions
- Answer questions about video content
- Generate video captions
Example usage (native mode for training):
Example usage (ONNX mode for inference only):
How It Works
InternVideo2 is a state-of-the-art video understanding model that combines:
- Video-text contrastive learning
- Masked video modeling
- Video-text generative learning
Reference: “InternVideo2: Scaling Video Foundation Models for Multimodal Video Understanding”
https://arxiv.org/abs/2403.15377
Constructors
| Constructor | Summary |
|---|---|
InternVideo2() |
Creates an InternVideo2 model using native layers for training and inference. |
InternVideo2(NeuralNetworkArchitecture<T> architecture, InternVideo2Options options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer, ILossFunction<T> lossFunction) |
|
InternVideo2(NeuralNetworkArchitecture<T> architecture, string onnxModelPath, InternVideo2Options options) |
Creates an InternVideo2 model using a pretrained ONNX model for inference. |
Properties
| Property | Summary |
|---|---|
bool SupportsTraining { get; } |
Gets whether training is supported (only in native mode). |
Methods
| Method | Summary |
|---|---|
ModelMetadata<T> GetModelMetadata() |
|
T ComputeSimilarity(Tensor<T> videoEmbedding, Tensor<T> textEmbedding) |
Computes similarity between video and text embeddings. |
Tensor<T> EncodeVideo(Tensor<T> videoFrames) |
Encodes video frames into an embedding vector. |