IEmotionRecognizer<T>
Interfaces · AiDotNet.Interfaces
Defines the contract for speech emotion recognition models.
Known implementations: Emotion2Vec<T>, HuBERTSER<T>, SpeechEmotionRecognizer<T>, Wav2Small<T>
For Beginners
This is like reading emotions from someone’s voice!
How humans convey emotion in speech:
- Pitch: Higher when excited/happy, lower when sad
- Speed: Faster when angry/excited, slower when sad
- Volume: Louder when angry, softer when sad/fearful
- Voice quality: Breathy, tense, relaxed
Applications:
- Call centers: Detect frustrated customers for escalation
- Mental health: Monitor patient emotional state
- Voice assistants: Respond appropriately to user mood
- Gaming: Adapt game difficulty/story based on player emotion
- Market research: Analyze focus group reactions
Challenges:
- Cultural differences in emotional expression
- Speaker variability (age, gender, accent)
- Context dependency (same words can mean different emotions)
- Mixed emotions (happy but nervous)
How It Works
Speech Emotion Recognition (SER) identifies emotional states from voice:
- Basic emotions: Happy, Sad, Angry, Fear, Surprise, Disgust, Neutral
- Arousal: Low (calm) to High (excited)
- Valence: Negative to Positive
- Dominance: Submissive to Dominant
Properties
| Property | Summary |
|---|---|
IReadOnlyList<string> SupportedEmotions { get; } |
Gets the list of emotions this model can detect. |
int SampleRate { get; } |
Gets the sample rate this recognizer operates at. |
Methods
| Method | Summary |
|---|---|
EmotionResult<T> RecognizeEmotion(Tensor<T> audio) |
Recognizes the primary emotion in speech audio. |
IReadOnlyDictionary<string, T> GetEmotionProbabilities(Tensor<T> audio) |
Gets probabilities for all supported emotions. |
IReadOnlyList<TimedEmotionResult<T>> RecognizeEmotionTimeSeries(Tensor<T> audio, int windowSizeMs, int hopSizeMs) |
Recognizes emotions over time (for longer recordings). |
T GetArousal(Tensor<T> audio) |
Gets arousal (activation) level from speech. |
T GetValence(Tensor<T> audio) |
Gets valence (positivity) level from speech. |
Vector<T> ExtractEmotionFeatures(Tensor<T> audio) |
Extracts emotion-relevant features from audio. |