OnsetsAndFrames<T>
Models & Types · AiDotNet.Audio.MusicAnalysis
Onsets and Frames piano transcription model from Google Magenta.
Inheritance: NeuralNetworkBase<T> → AudioNeuralNetworkBase<T> → OnsetsAndFrames<T>
Implements: IMusicTranscriber<T>
For Beginners
Onsets and Frames is a piano-specific music transcriber. It listens to piano music and detects every key press: when each key is pressed (onset), how long it is held (frame), and which key it is (pitch). The output is a list of notes that can be saved as MIDI or displayed as sheet music.
Usage:
How It Works
Onsets and Frames (Hawthorne et al., 2018) jointly predicts note onsets and frame-level activations for automatic piano transcription. The model uses CNN acoustic features with bidirectional LSTMs, trained on the MAESTRO dataset. It achieves ~90% note F1 on piano.
Constructors
| Constructor | Summary |
|---|---|
OnsetsAndFrames(NeuralNetworkArchitecture<T> architecture, OnsetsAndFramesOptions options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer) |
Creates an Onsets and Frames model in native training mode. |
OnsetsAndFrames(NeuralNetworkArchitecture<T> architecture, string modelPath, OnsetsAndFramesOptions options) |
Creates an Onsets and Frames model in native training mode. |
Properties
| Property | Summary |
|---|---|
int MidiOffset { get; } |
|
int NumMidiNotes { get; } |
Methods
| Method | Summary |
|---|---|
IReadOnlyList<TranscribedNote<T>> ExtractNotes(Tensor<T> frameActivations, Tensor<T> onsetActivations, double frameThreshold, double onsetThreshold) |
|
IReadOnlyList<TranscribedNote<T>> Transcribe(Tensor<T> audio) |
|
ModelMetadata<T> GetModelMetadata() |
|
Task<IReadOnlyList<TranscribedNote<T>>> TranscribeAsync(Tensor<T> audio, CancellationToken cancellationToken) |
|
Tensor<T> GetFrameActivations(Tensor<T> audio) |
|
Tensor<T> GetOnsetActivations(Tensor<T> audio) |