IMusicSourceSeparator<T>
Interfaces · AiDotNet.Interfaces
Interface for music source separation models that isolate individual instruments/vocals from a mix.
Implements: IFeatureAware, IFullModel<T, Tensor<T>, Tensor<T>>, IGradientComputable<T, Tensor<T>, Tensor<T>>
Known implementations: BSRoFormer<T>, BandSplitRNN<T>, DannaSep<T>, HTDemucs<T>, MelBandRoFormer<T>, MusicSourceSeparator<T>, SCNet<T>
For Beginners
Source separation is like un-mixing a smoothie back into its original fruits.
How it works:
- The mixed audio is converted to a spectrogram
- A neural network learns which parts belong to which source
- Masks are applied to isolate each source
- Individual spectrograms are converted back to audio
Common separations:
- 2-stem: Vocals vs Accompaniment
- 4-stem: Vocals, Drums, Bass, Other
- 5-stem: Vocals, Drums, Bass, Piano, Other
Use cases:
- Karaoke (remove vocals)
- Remixing (isolate and rearrange parts)
- Music transcription (analyze individual instruments)
- Sample extraction (get drum loops, vocal hooks)
- Music education (practice with isolated parts)
Popular models:
- Demucs (Facebook/Meta)
- Spleeter (Deezer)
- Open-Unmix
How It Works
Music source separation (also called audio source separation or “unmixing”) takes a mixed audio signal and separates it into individual components like vocals, drums, bass, and other instruments.
This interface extends IFullModel for Tensor-based audio processing.
Properties
| Property | Summary |
|---|---|
IReadOnlyList<string> SupportedSources { get; } |
Gets the sources this model can separate. |
bool IsOnnxMode { get; } |
Gets whether this model is running in ONNX inference mode. |
int NumStems { get; } |
Gets the number of stems/sources this model produces. |
int SampleRate { get; } |
Gets the expected sample rate for input audio. |
Methods
| Method | Summary |
|---|---|
SourceSeparationResult<T> Separate(Tensor<T> audio) |
Separates all sources from the audio mix. |
Task<SourceSeparationResult<T>> SeparateAsync(Tensor<T> audio, CancellationToken cancellationToken) |
Separates all sources asynchronously. |
Tensor<T> ExtractSource(Tensor<T> audio, string source) |
Extracts a specific source from the mix. |
Tensor<T> GetSourceMask(Tensor<T> audio, string source) |
Gets the soft mask for a specific source. |
Tensor<T> Remix(SourceSeparationResult<T> separationResult, IReadOnlyDictionary<string, double> sourceVolumes) |
Remixes the separated sources with custom volumes. |
Tensor<T> RemoveSource(Tensor<T> audio, string source) |
Removes a specific source from the mix. |