Emotion2Vec<T>
Models & Types · AiDotNet.Audio.Emotion
emotion2vec universal speech emotion recognition model.
Inheritance: NeuralNetworkBase<T> → AudioNeuralNetworkBase<T> → AudioClassifierBase<T> → Emotion2Vec<T>
Implements: IEmotionRecognizer<T>
For Beginners
emotion2vec detects emotions from speech. Feed it audio of someone talking and it tells you what emotion they’re expressing (happy, sad, angry, etc.) along with a confidence score. It can also measure arousal (calm vs excited) and valence (negative vs positive).
Usage:
How It Works
emotion2vec (Ma et al., 2023) is a universal speech emotion representation model using self-supervised pre-training on unlabeled speech followed by fine-tuning. It achieves state-of-the-art results across multiple SER benchmarks with a single model.
Constructors
| Constructor | Summary |
|---|---|
Emotion2Vec(NeuralNetworkArchitecture<T> architecture, Emotion2VecOptions options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer) |
Creates an emotion2vec model in native training mode. |
Emotion2Vec(NeuralNetworkArchitecture<T> architecture, string modelPath, Emotion2VecOptions options) |
Creates an emotion2vec model in native training mode. |
Properties
| Property | Summary |
|---|---|
IReadOnlyList<string> SupportedEmotions { get; } |
Methods
| Method | Summary |
|---|---|
EmotionResult<T> RecognizeEmotion(Tensor<T> audio) |
|
IReadOnlyDictionary<string, T> GetEmotionProbabilities(Tensor<T> audio) |
|
IReadOnlyList<TimedEmotionResult<T>> RecognizeEmotionTimeSeries(Tensor<T> audio, int windowSizeMs, int hopSizeMs) |
|
ModelMetadata<T> GetModelMetadata() |
|
T GetArousal(Tensor<T> audio) |
|
T GetValence(Tensor<T> audio) |
|
Tensor<T> ForwardForTraining(Tensor<T> input) |
Runs the native stack for tape training and pools frame logits into one embedding. |
Vector<T> ExtractEmotionFeatures(Tensor<T> audio) |