PANNs<T>
Models & Types · AiDotNet.Audio.Classification
PANNs (Pre-trained Audio Neural Networks) CNN14 model for audio classification.
Inheritance: NeuralNetworkBase<T> → AudioNeuralNetworkBase<T> → AudioClassifierBase<T> → PANNs<T>
Implements: IAudioEventDetector<T>
For Beginners
PANNs is like having increasingly specialized detectors stacked on top of each other. The first layers detect simple patterns (edges, tones), middle layers detect intermediate patterns (harmonics, rhythms), and final layers detect complete sounds (speech, music, dog bark). Being CNN-based, it is faster than Transformer models but slightly less accurate.
Usage:
How It Works
PANNs (Kong et al., IEEE/ACM TASLP 2020) provides pre-trained CNN-based audio classification models. The flagship CNN14 achieves 43.1% mAP on AudioSet-2M and is widely used as a feature extractor for downstream audio tasks.
Architecture: CNN14 is a 14-layer convolutional neural network:
- Input: 64-bin log-mel spectrogram at 32 kHz
- 6 CNN blocks: Each with two 3x3 conv layers, batch norm, ReLU, and 2x2 avg pooling
- Channel progression: 64 -> 128 -> 256 -> 512 -> 1024 -> 2048
- Global pooling: Average and max pooling combined
- Classification head: 2048-dim FC -> 527-class sigmoid output
References:
- Paper: “PANNs: Large-Scale Pretrained Audio Neural Networks” (Kong et al., 2020)
- Repository:
https://github.com/qiuqiangkong/audioset_tagging_cnn
Constructors
| Constructor | Summary |
|---|---|
PANNs(NeuralNetworkArchitecture<T> architecture, PANNsOptions options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer, ILossFunction<T> lossFunction) |
|
PANNs(NeuralNetworkArchitecture<T> architecture, string modelPath, PANNsOptions options) |
Properties
| Property | Summary |
|---|---|
IReadOnlyList<string> EventLabels { get; } |
|
IReadOnlyList<string> SupportedEvents { get; } |
|
double TimeResolution { get; } |
Methods
| Method | Summary |
|---|---|
AudioEventResult<T> Detect(Tensor<T> audio) |
|
AudioEventResult<T> Detect(Tensor<T> audio, T threshold) |
|
AudioEventResult<T> DetectSpecific(Tensor<T> audio, IReadOnlyList<string> types) |
|
AudioEventResult<T> DetectSpecific(Tensor<T> audio, IReadOnlyList<string> types, T threshold) |
|
Dictionary<string, Tensor<T>> GetNamedLayerActivations(Tensor<T> input) |
|
IStreamingEventDetectionSession<T> StartStreamingSession() |
|
IStreamingEventDetectionSession<T> StartStreamingSession(int sr, T thr) |
|
ModelMetadata<T> GetModelMetadata() |
|
Task<AudioEventResult<T>> DetectAsync(Tensor<T> audio, CancellationToken ct) |
|
Tensor<T> ForwardForTraining(Tensor<T> input) |
|
Tensor<T> GetEventProbabilities(Tensor<T> audio) |
Fields
| Field | Summary |
|---|---|
string[] AudioSetLabels |