MT3Options

Options & Configuration ยท AiDotNet.Audio.MusicAnalysis

Configuration options for the MT3 (Multi-Track Music Transcription) model.

Inheritance: ModelOptions โ†’ MT3Options

For Beginners

MT3 listens to a full song with multiple instruments and writes out the sheet music (as MIDI) for each instrument separately. It can tell which notes the piano is playing while also transcribing the guitar, drums, and bass at the same time.

How It Works

MT3 (Gardner et al., 2022, Google) is a Transformer-based model that transcribes polyphonic audio into MIDI across multiple instruments simultaneously. It uses a T5-style encoder-decoder architecture with spectrogram input and tokenized MIDI output.

Constructors

Constructor Summary
MT3Options()

Properties

Property Summary
OnnxModelOptions OnnxOptions { get; set; } Gets or sets the ONNX runtime options.
double DropoutRate { get; set; } Gets or sets the dropout rate.
double LearningRate { get; set; } Gets or sets the learning rate.
int DecoderDim { get; set; } Gets or sets the decoder hidden dimension.
int EncoderDim { get; set; } Gets or sets the encoder hidden dimension.
int FftSize { get; set; } Gets or sets the FFT window size.
int HopLength { get; set; } Gets or sets the hop length between frames.
int MaxInstruments { get; set; } Gets or sets the maximum number of instruments.
int NumAttentionHeads { get; set; } Gets or sets the number of attention heads.
int NumDecoderLayers { get; set; } Gets or sets the number of decoder layers.
int NumEncoderLayers { get; set; } Gets or sets the number of encoder layers.
int NumMels { get; set; } Gets or sets the number of mel filterbank channels.
int SampleRate { get; set; } Gets or sets the expected audio sample rate in Hz.
int VocabSize { get; set; } Gets or sets the MIDI vocabulary size (tokens).
string ModelPath { get; set; } Gets or sets the path to the ONNX model file.