AudioLDMClassifier<T>
Models & Types · AiDotNet.Audio.Classification
AudioLDM Classifier that repurposes AudioLDM’s latent representations for audio event detection.
Inheritance: NeuralNetworkBase<T> → AudioNeuralNetworkBase<T> → AudioClassifierBase<T> → AudioLDMClassifier<T>
Implements: IAudioEventDetector<T>
For Beginners
AudioLDM was originally built to create audio from text descriptions. But it turns out the “understanding” it developed during training is also great for recognizing sounds. This classifier reuses that understanding: instead of generating audio, it identifies what sounds are present in a recording.
Usage:
How It Works
AudioLDM Classifier (Liu et al., 2023) extracts intermediate features from the AudioLDM diffusion U-Net to build a strong audio classifier. Since AudioLDM was trained to generate audio from text, its internal representations capture rich, semantically meaningful audio features that transfer well to classification tasks.
Constructors
| Constructor | Summary |
|---|---|
AudioLDMClassifier(NeuralNetworkArchitecture<T> architecture, AudioLDMClassifierOptions options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer) |
Creates an AudioLDM Classifier in native training mode. |
AudioLDMClassifier(NeuralNetworkArchitecture<T> architecture, string modelPath, AudioLDMClassifierOptions options) |
Creates an AudioLDM Classifier in native training mode. |
Properties
| Property | Summary |
|---|---|
IReadOnlyList<string> StandardLabels { get; } |
Standard AudioSet labels for audio event detection. |
IReadOnlyList<string> SupportedEvents { get; } |
|
bool IsOnnxMode { get; } |
|
double TimeResolution { get; } |
Methods
| Method | Summary |
|---|---|
AudioEventResult<T> Detect(Tensor<T> audio) |
|
AudioEventResult<T> Detect(Tensor<T> audio, T threshold) |
|
AudioEventResult<T> DetectSpecific(Tensor<T> audio, IReadOnlyList<string> eventTypes) |
|
AudioEventResult<T> DetectSpecific(Tensor<T> audio, IReadOnlyList<string> eventTypes, T threshold) |
|
IStreamingEventDetectionSession<T> StartStreamingSession() |
|
IStreamingEventDetectionSession<T> StartStreamingSession(int sampleRate, T threshold) |
|
ModelMetadata<T> GetModelMetadata() |
|
Task<AudioEventResult<T>> DetectAsync(Tensor<T> audio, CancellationToken cancellationToken) |
|
Tensor<T> GetEventProbabilities(Tensor<T> audio) |