Data2Vec2<T>
Models & Types · AiDotNet.Audio.Foundations
data2vec 2.0 self-supervised audio representation model.
Inheritance: NeuralNetworkBase<T> → AudioNeuralNetworkBase<T> → Data2Vec2<T>
Implements: IAudioFoundationModel<T>
For Beginners
data2vec 2.0 learns audio features by predicting its own hidden representations - like learning by explaining things to yourself. Unlike HuBERT which predicts discrete labels, data2vec predicts rich continuous features. It can be used as a powerful feature extractor for any downstream audio task.
Usage:
How It Works
data2vec 2.0 (Baevski et al., 2023, Meta) is a self-supervised framework that predicts contextualized latent representations rather than modality-specific targets. Version 2.0 is 16x faster than v1 through efficient data encoding and self-distillation. It achieves strong results on speech, vision, and language tasks with the same method.
Constructors
| Constructor | Summary |
|---|---|
Data2Vec2(NeuralNetworkArchitecture<T> architecture, Data2Vec2Options options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer) |
Creates a data2vec 2.0 model in native training mode. |
Data2Vec2(NeuralNetworkArchitecture<T> architecture, string modelPath, Data2Vec2Options options) |
Creates a data2vec 2.0 model in native training mode. |
Properties
| Property | Summary |
|---|---|
int EmbeddingDimension { get; } |
|
int NumLayers { get; } |
Methods
| Method | Summary |
|---|---|
ModelMetadata<T> GetModelMetadata() |
|
Task<Tensor<T>> ExtractEmbeddingsAsync(Tensor<T> audio, CancellationToken cancellationToken) |
|
Tensor<T> ExtractEmbeddings(Tensor<T> audio) |
|
Tensor<T> ExtractLayerFeatures(Tensor<T> audio, int layerIndex) |
|
Tensor<T> ExtractWeightedFeatures(Tensor<T> audio, T[] layerWeights) |