ACEStep<T> |
ACE-Step accelerated consistency-enhanced music generation model. |
ASTModel<T> |
AST (Audio Spectrogram Transformer, Gong et al. |
AST<T> |
AST (Audio Spectrogram Transformer) model for audio event detection and classification. |
AudioEvent |
Represents a detected audio event. |
AudioEventDetector<T> |
Audio event detection model for identifying sounds in audio (AudioSet-style). |
AudioFlamingo2<T> |
Audio Flamingo 2 multimodal audio-language model for audio understanding with interleaved inputs. |
AudioGenModel<T> |
AudioGen model for generating audio from text descriptions using neural audio codecs. |
AudioGenResult<T> |
Result of audio generation. |
AudioLDMClassifier<T> |
AudioLDM Classifier that repurposes AudioLDM’s latent representations for audio event detection. |
AudioLM<T> |
AudioLM: audio generation as language modeling over semantic and acoustic tokens (Borsos et al. |
AudioMAE<T> |
Audio-MAE (Masked Autoencoders for Audio) model for audio classification. |
AudioSep<T> |
AudioSep - foundation model for open-vocabulary audio separation and sound event detection (Liu et al., ICML 2024). |
AudioSuperResolution<T> |
Audio Super-Resolution model for upsampling low-resolution audio to high-resolution (Kuleshov et al., 2017; Li et al., 2021). |
BEATs<T> |
BEATs (Audio Pre-Training with Acoustic Tokenizers) model for state-of-the-art audio event detection and classification. |
BSRoFormer<T> |
BS-RoFormer (Band-Split Rotary Transformer) for music source separation. |
BandSplitRNNEnhancer<T> |
Band-Split RNN speech enhancement model (Luo and Yu, 2023). |
BandSplitRNN<T> |
BandSplitRNN for music source separation (Luo and Yu, 2023). |
BasicPitch<T> |
Basic Pitch polyphonic music transcription model from Spotify. |
BeatTracker<T> |
Extracts beat and tempo information from audio. |
CAMPlusPlus<T> |
CAM++ (Context-Aware Masking Plus Plus) speaker verification model (Wang et al., 2023). |
CLAPModel<T> |
CLAP (Contrastive Language-Audio Pretraining) — a dual-encoder neural network that learns to align audio and text representations in a shared embedding space via a contrastive objective. |
CLAP<T> |
CLAP (Contrastive Language-Audio Pre-training) as an audio classifier: zero-shot against text prompts, or fine-tuned with a linear head. |
CMGAN<T> |
CMGAN (Conformer-based Metric GAN) for speech enhancement. |
CREPE<T> |
CREPE (Convolutional Representation for Pitch Estimation) neural pitch detector. |
CRNNEventDetector<T> |
CRNN (Convolutional Recurrent Neural Network) model for Sound Event Detection. |
CTCDecoder<T> |
CTC (Connectionist Temporal Classification) decoder-based speech recognition model. |
Canary<T> |
Canary multilingual speech recognition and translation model from NVIDIA. |
ChordRecognizer<T> |
Recognizes chords from audio using chromagram analysis. |
ChordSegment<T> |
Represents a chord segment in audio. |
ChromaExtractor<T> |
Extracts chromagram (pitch class profile) features from audio signals. |
ChromaprintFingerprinter<T> |
Chromaprint-style audio fingerprinter based on chroma features. |
Compressor<T> |
Dynamic range compressor effect. |
ConformerFP<T> |
Conformer-based audio fingerprinting model combining self-attention with convolutions. |
Conformer<T> |
Conformer (Gulati et al. |
ConstantQTransform<T> |
Constant-Q Transform (CQT) for music analysis with logarithmic frequency resolution. |
ConvTasNet<T> |
Conv-TasNet: A fully-convolutional time-domain audio separation network. |
DAC<T> |
DAC, the Descript Audio Codec: a Snake-activated convolutional encoder–decoder with a residual vector quantizer of factorized, L2-normalized codes, trained with quantizer dropout against multi-period and multi-band multi-scale STFT discrimi… |
DCCRN<T> |
DCCRN - Deep Complex Convolution Recurrent Network for speech enhancement. |
DannaSep<T> |
Danna-Sep music source separation model using dual-path attention networks. |
Data2Vec2<T> |
data2vec 2.0 self-supervised audio representation model. |
DeepFilterNet<T> |
DeepFilterNet - State-of-the-art deep filtering network for speech enhancement. |
DemucsNoise<T> |
Demucs for Noise - real-time noise suppression using the Demucs architecture (Defossez et al., 2020, Meta). |
DiarizationResult |
Result of speaker diarization. |
EAT<T> |
EAT (Efficient Audio Transformer) model for efficient audio event detection and classification. |
ECAPATDNNLanguageIdentifier<T> |
ECAPA-TDNN (Emphasized Channel Attention, Propagation and Aggregation Time Delay Neural Network) for spoken language identification. |
ECAPATDNNSpeaker<T> |
ECAPA-TDNN speaker verification and embedding extraction model. |
Emotion2Vec<T> |
emotion2vec universal speech emotion recognition model. |
EnCodec<T> |
EnCodec: a streaming convolutional encoder–decoder with an LSTM, residual vector quantization of its latent, and adversarial training against a multi-scale STFT discriminator with a loss balancer. |
EncodedFrame<T> |
One encoded chunk: its codes [n_q, frames] and its volume scale (1 when not normalized). |
EnergyBasedVad<T> |
Simple energy-based voice activity detector (algorithmic, no neural network). |
EqBand<T> |
Represents a single EQ band with biquad filter. |
FDYSED<T> |
FDY-SED (Frequency Dynamic Sound Event Detection) model for DCASE-winning SED. |
FRCRN<T> |
FRCRN (Frequency Recurrence CRN) speech enhancement model (Zhao et al., ICASSP 2022). |
FastConformer<T> |
Fast Conformer speech recognition model (Rekesh et al., 2023, NVIDIA NeMo). |
FullSubNetPlus<T> |
FullSubNet+ (Full-Band and Sub-Band Fusion Network Plus) for speech enhancement. |
GenreClassificationResult |
Result of genre classification. |
GenreClassifier<T> |
Music genre classification after Tzanetakis and Cook 2002: a 30-dimensional “musical surface” feature vector per recording, classified by one diagonal-covariance Gaussian mixture per genre. |
GenreFeatures |
Features extracted for genre classification. |
HTDemucs<T> |
HTDemucs (Hybrid Transformer Demucs) for music source separation. |
HTSAT<T> |
HTS-AT (Hierarchical Token-Semantic Audio Transformer) model for efficient audio classification. |
HuBERTSER<T> |
HuBERT-SER (HuBERT for Speech Emotion Recognition) model. |
HuBERT<T> |
HuBERT (Hidden-Unit BERT) self-supervised speech representation model. |
IdentificationResult |
Result of speaker identification. |
KeyDetectionResult<T> |
Result of key detection. |
KeyDetector<T> |
Detects the musical key of audio using chromagram analysis. |
LocalizationResult |
Result of sound source localization. |
MERT<T> |
MERT self-supervised music understanding foundation model. |
MPSENet<T> |
MP-SENet (Multi-Path Speech Enhancement Network) model (Lu et al., 2023). |
MT3<T> |
MT3 (Multi-Track Music Transcription) model using T5-style encoder-decoder architecture. |
MadmomBeatTracker<T> |
Madmom-style neural beat tracker using bidirectional RNNs. |
MarbleNet<T> |
MarbleNet lightweight 1D separable convolutional VAD model (NVIDIA NeMo). |
MelBandRoFormer<T> |
MelBand-RoFormer for mel-band music source separation. |
MelodyExtractor<T> |
Neural Melody Extractor that identifies the primary melodic line from polyphonic audio. |
MfccExtractor<T> |
Extracts Mel-Frequency Cepstral Coefficients (MFCCs) from audio signals. |
MusicFlamingo<T> |
Music Flamingo multimodal music-language model for music understanding and reasoning. |
MusicGenModel<T> |
MusicGen (Copet et al. |
MusicGenTrainingSample<T> |
One MusicGen training example: a music clip, its text description and optional metadata. |
MusicSegment<T> |
Represents a labeled segment of music structure. |
MusicSourceSeparator<T> |
Music source separation model for separating audio into stems (vocals, drums, bass, other). |
MusicStructureAnalyzer<T> |
Music Structure Analyzer that segments songs into structural sections (intro, verse, chorus, etc.). |
MusicTaggingTransformer<T> |
Music Tagging Transformer for multi-label music tag prediction (genre, mood, instrument, era). |
NeuralNoiseReducer<T> |
Neural network-based noise reducer for high-quality audio enhancement. |
NeuralParametricEQ<T> |
Neural Parametric EQ model for automatic equalization (Steinmetz et al., 2022). |
OnsetsAndFrames<T> |
Onsets and Frames piano transcription model from Google Magenta. |
PANNsModel<T> |
PANNs (Pretrained Audio Neural Networks) audio classifier — a CNN14-style convolutional backbone over log-mel spectrograms, trained for AudioSet tagging (Kong et al. |
PANNs<T> |
PANNs (Pre-trained Audio Neural Networks) CNN14 model for audio classification. |
ParametricEqualizer<T> |
Multi-band parametric equalizer effect. |
PeakNetFP<T> |
PeakNetFP spectral peak-based neural audio fingerprinting model. |
Pengi<T> |
Pengi audio language model that frames all audio tasks as text-generation. |
PyAnnote<T> |
pyannote 3.x end-to-end speaker diarization model. |
QuailVad<T> |
Quail VAD - lightweight voice activity detection optimized for on-device deployment. |
Qwen2Audio<T> |
Qwen2-Audio multimodal audio-language model for audio understanding and reasoning. |
RNNTransducer<T> |
RNN-Transducer (RNN-T) streaming speech recognition model. |
Result |
The decoded audio: interleaved samples in [−1, 1], the sample rate and the channel count. |
Reverb<T> |
Algorithmic reverb effect using Schroeder-Moorer structure. |
RoomImpulseResponse<T> |
FiNS — Filtered Noise Shaping network that estimates a time-domain room impulse response directly from reverberant speech. |
SALMONN<T> |
SALMONN dual-encoder audio-language model for speech and audio understanding. |
SCNet<T> |
SCNet (Sparse Compression Network) for music source separation (Tong et al., 2024). |
SceneClassificationResult |
Result of acoustic scene classification. |
SceneClassifier<T> |
Acoustic scene classification model for identifying recording environments. |
SceneFeatures |
Features extracted for scene classification. |
SeparationResult<T> |
Result of music source separation containing individual stems. |
SileroVad<T> |
Silero Voice Activity Detection model - high accuracy neural network VAD. |
SoundLocalizer<T> |
Sound source localization using microphone arrays. |
SoundStream<T> |
SoundStream: a fully convolutional, causal encoder–decoder whose embeddings are compressed by a residual vector quantizer trained with quantizer dropout, so one model serves every bitrate up to its maximum, trained adversarially against a w… |
SpeakerDiarizer<T> |
Performs speaker diarization (who spoke when) on audio recordings. |
SpeakerEmbeddingExtractor<T> |
Extracts speaker embeddings (d-vectors) from audio for speaker recognition. |
SpeakerEmbedding<T> |
Represents a speaker embedding vector. |
SpeakerLM<T> |
SpeakerLM language-model-based speaker diarization and verification model. |
SpeakerMatch |
A speaker match with score. |
SpeakerTurn |
Represents a speaker turn in diarization output. |
SpeakerVerifier<T> |
Verifies speaker identity by comparing embeddings against enrolled speakers. |
SpectralFeatureExtractor<T> |
Extracts spectral features from audio signals including centroid, bandwidth, rolloff, and flux. |
SpectralSubtractionEnhancer<T> |
Audio enhancer using spectral subtraction for noise reduction. |
SpectrogramFingerprinter<T> |
Spectrogram peak-based audio fingerprinter (Shazam-style). |
SpeechEmotionRecognizer<T> |
Neural network-based speech emotion recognition model that classifies emotional states from audio. |
SpeechTokenizer<T> |
SpeechTokenizer: EnCodec’s encoder–quantizer–decoder with a BiLSTM encoder, whose first residual codebook is distilled toward a self-supervised teacher’s representations so that it carries the content of speech and the remaining codebooks t… |
SpikingFullSubNet<T> |
Spiking-FullSubNet speech enhancement model using spiking neural networks. |
StableAudioModel<T> |
Stable Audio model for generating high-quality audio from text descriptions. |
TFGridNet<T> |
TF-GridNet (Time-Frequency GridNet) for speech enhancement and separation. |
Tempogram<T> |
Neural Tempogram model for tempo estimation over time. |
TitaNet<T> |
TitaNet speaker verification and embedding extraction model. |
VerificationResult |
Result of speaker verification. |
VoxLingua107Identifier<T> |
VoxLingua107 language identifier supporting 107 languages. |
W2vBert<T> |
w2v-BERT: self-supervised speech representations from contrastive learning and masked prediction together (Chung et al. |
Wav2Small<T> |
Wav2Small lightweight speech emotion recognition model (Gomez-Alanis et al., 2024). |
Wav2Vec2LanguageIdentifier<T> |
Wav2Vec2 model fine-tuned for spoken language identification. |
Wav2Vec2Model<T> |
Wav2Vec2 self-supervised speech recognition model. |
Wav2Vec2<T> |
wav2vec 2.0 self-supervised speech representation model. |
WavLMSpeaker<T> |
WavLM-based speaker verification and embedding extraction model (Chen et al., 2022). |
WavLM<T> |
WavLM self-supervised speech representation model from Microsoft. |
WebRTCVad<T> |
Neural WebRTC VAD model for low-latency voice activity detection. |
WhisperModel<T> |
Whisper automatic speech recognition model for transcribing audio to text. |
WhisperResult |
Result of Whisper transcription. |
WhisperSegment |
A segment of transcribed speech with timing. |
WhisperTokenizer |
Tokenizer for Whisper speech recognition model. |
WhisperWord |
A transcribed word with timing information. |
WorldPitchDetector<T> |
F0 estimation with WORLD’s DIO (Distributed Inline-filter Operation), refined by StoneMask. |
YinPitchDetector<T> |
YIN pitch detection algorithm implementation. |
YuE<T> |
YuE full-song music generation model with vocals and accompaniment. |
Zipformer<T> |
Zipformer speech recognition model (Yao et al., 2023, Next-gen Kaldi). |