MusicSourceSeparator<T>

Models & Types · AiDotNet.Audio.SourceSeparation

Music source separation model for separating audio into stems (vocals, drums, bass, other).

Inheritance: NeuralNetworkBase<T> → AudioNeuralNetworkBase<T> → MusicSourceSeparator<T>

Implements: IMusicSourceSeparator<T>

For Beginners

Source separation is like unmixing a smoothie:

  • Input: Mixed audio with multiple instruments and vocals
  • Output: Separate tracks for vocals, drums, bass, and other instruments
  • Uses neural networks to predict which parts of the spectrum belong to each source

Usage with ONNX model:

Usage for training:

How It Works

This implements a U-Net based source separation approach similar to Spleeter/Demucs. The model separates mixed audio into individual instrument stems using spectral masking.

Constructors

Constructor Summary
MusicSourceSeparator(NeuralNetworkArchitecture<T> architecture, SourceSeparationOptions options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer) Creates a MusicSourceSeparator for native training mode.
MusicSourceSeparator(SourceSeparationOptions options) Creates a MusicSourceSeparator for CPU-based spectral processing.
MusicSourceSeparator(string modelPath, SourceSeparationOptions options) Creates a MusicSourceSeparator for ONNX inference mode.

Properties

Property Summary
IReadOnlyList<string> SupportedSources { get; } Gets the sources this model can separate.
int NumStems { get; } Gets the number of stems/sources this model produces.

Methods

Method Summary
Dictionary<string, Tensor<T>> GetNamedLayerActivations(Tensor<T> input) Captures per-stage activations of the waveform-Demucs forward pass.
ModelInputShapeConstraint GetInputShapeConstraint() Generated from [ModelInputShapeConstraint].
ModelMetadata<T> GetModelMetadata() Gets model metadata for serialization.
MusicSourceSeparator<T> CreateCpuOnly(SourceSeparationOptions options) Creates a MusicSourceSeparator for CPU-based spectral processing without neural network.
SourceSeparationResult<T> Separate(Tensor<T> audio) Separates all sources from the audio mix.
Task<SourceSeparationResult<T>> SeparateAsync(Tensor<T> audio, CancellationToken cancellationToken) Separates all sources asynchronously.
Tensor<T> ExtractSource(Tensor<T> audio, string source) Extracts a specific source from the mix.
Tensor<T> ForwardForTraining(Tensor<T> input) Training forward pass.
Tensor<T> GetSourceMask(Tensor<T> audio, string source) Gets the soft mask for a specific source.
Tensor<T> Remix(SourceSeparationResult<T> separationResult, IReadOnlyDictionary<string, double> sourceVolumes) Remixes the separated sources with custom volumes.
Tensor<T> RemoveSource(Tensor<T> audio, string source) Removes a specific source from the mix.

Fields

Field Summary
string[] FiveStemSources Source names for 5-stem separation.
string[] StandardSources Standard source names for 4-stem separation.
string[] TwoStemSources Source names for 2-stem separation.