MusicTaggingTransformer<T>

Models & Types · AiDotNet.Audio.MusicAnalysis

Music Tagging Transformer for multi-label music tag prediction (genre, mood, instrument, era).

Inheritance: NeuralNetworkBase<T> → AudioNeuralNetworkBase<T> → MusicTaggingTransformer<T>

For Beginners

This model listens to music and automatically tags it with descriptive labels - like “rock”, “upbeat”, “guitar”, “1980s”, or “relaxing”. It’s the technology behind automatic music categorization in streaming services like Spotify’s genre detection.

Usage:

How It Works

The Music Tagging Transformer (Won et al., 2021) uses a Transformer encoder on mel spectrogram features to predict music tags (genre, mood, instrument, era). It achieves state-of-the-art results on the MagnaTagATune and Million Song Dataset benchmarks.

Constructors

Constructor Summary
MusicTaggingTransformer(NeuralNetworkArchitecture<T> architecture, MusicTaggingTransformerOptions options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer) Creates a Music Tagging Transformer in native training mode.
MusicTaggingTransformer(NeuralNetworkArchitecture<T> architecture, string modelPath, MusicTaggingTransformerOptions options) Creates a Music Tagging Transformer in native training mode.

Properties

Property Summary
int NumTags { get; } Gets the number of tags.
string[] TagLabels { get; } Gets the tag labels this model can predict.

Methods

Method Summary
IReadOnlyList<ValueTuple<string, T>> GetTopKTags(Tensor<T> audio, int k) Gets the top-K tags by confidence.
IReadOnlyList<ValueTuple<string, T>> PredictTags(Tensor<T> audio, double threshold) Predicts music tags for the given audio.
ModelMetadata<T> GetModelMetadata()
Task<IReadOnlyList<ValueTuple<string, T>>> PredictTagsAsync(Tensor<T> audio, double threshold, CancellationToken cancellationToken) Predicts music tags asynchronously.
Tensor<T> GetTagProbabilities(Tensor<T> audio) Gets raw tag probabilities for all tags.