BasicPitch<T>

Models & Types · AiDotNet.Audio.MusicAnalysis

Basic Pitch polyphonic music transcription model from Spotify.

Inheritance: NeuralNetworkBase<T> → AudioNeuralNetworkBase<T> → BasicPitch<T>

Implements: IMusicTranscriber<T>

For Beginners

Basic Pitch is like a music-to-MIDI converter. You feed it a recording of music (even with multiple instruments playing at once) and it outputs a list of every note that was played, when it started, when it stopped, and how loud it was. This is called “polyphonic transcription” because it handles multiple notes at the same time.

Usage:

How It Works

Basic Pitch (Bittner et al., 2022) is a lightweight CNN for polyphonic music transcription. It produces three outputs: note activations, onset activations, and pitch contour. Combined, these produce MIDI-like note events. The model is fast enough for real-time use and handles multiple simultaneous instruments.

Constructors

Constructor Summary
BasicPitch(NeuralNetworkArchitecture<T> architecture, BasicPitchOptions options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer) Creates a Basic Pitch model in native training mode.
BasicPitch(NeuralNetworkArchitecture<T> architecture, string modelPath, BasicPitchOptions options) Creates a Basic Pitch model in native training mode.

Properties

Property Summary
int MidiOffset { get; }
int NumMidiNotes { get; }

Methods

Method Summary
IReadOnlyList<TranscribedNote<T>> ExtractNotes(Tensor<T> frameActivations, Tensor<T> onsetActivations, double frameThreshold, double onsetThreshold)
IReadOnlyList<TranscribedNote<T>> Transcribe(Tensor<T> audio)
ModelMetadata<T> GetModelMetadata()
Task<IReadOnlyList<TranscribedNote<T>>> TranscribeAsync(Tensor<T> audio, CancellationToken cancellationToken)
Tensor<T> GetFrameActivations(Tensor<T> audio)
Tensor<T> GetOnsetActivations(Tensor<T> audio)