MusicSourceSeparator<T>
Models & Types · AiDotNet.Audio.SourceSeparation
Music source separation model for separating audio into stems (vocals, drums, bass, other).
Inheritance: NeuralNetworkBase<T> → AudioNeuralNetworkBase<T> → MusicSourceSeparator<T>
Implements: IMusicSourceSeparator<T>
For Beginners
Source separation is like unmixing a smoothie:
- Input: Mixed audio with multiple instruments and vocals
- Output: Separate tracks for vocals, drums, bass, and other instruments
- Uses neural networks to predict which parts of the spectrum belong to each source
Usage with ONNX model:
Usage for training:
How It Works
This implements a U-Net based source separation approach similar to Spleeter/Demucs. The model separates mixed audio into individual instrument stems using spectral masking.
Constructors
| Constructor | Summary |
|---|---|
MusicSourceSeparator(NeuralNetworkArchitecture<T> architecture, SourceSeparationOptions options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer) |
Creates a MusicSourceSeparator for native training mode. |
MusicSourceSeparator(SourceSeparationOptions options) |
Creates a MusicSourceSeparator for CPU-based spectral processing. |
MusicSourceSeparator(string modelPath, SourceSeparationOptions options) |
Creates a MusicSourceSeparator for ONNX inference mode. |
Properties
| Property | Summary |
|---|---|
IReadOnlyList<string> SupportedSources { get; } |
Gets the sources this model can separate. |
int NumStems { get; } |
Gets the number of stems/sources this model produces. |
Methods
| Method | Summary |
|---|---|
Dictionary<string, Tensor<T>> GetNamedLayerActivations(Tensor<T> input) |
Captures per-stage activations of the waveform-Demucs forward pass. |
ModelInputShapeConstraint GetInputShapeConstraint() |
Generated from [ModelInputShapeConstraint]. |
ModelMetadata<T> GetModelMetadata() |
Gets model metadata for serialization. |
MusicSourceSeparator<T> CreateCpuOnly(SourceSeparationOptions options) |
Creates a MusicSourceSeparator for CPU-based spectral processing without neural network. |
SourceSeparationResult<T> Separate(Tensor<T> audio) |
Separates all sources from the audio mix. |
Task<SourceSeparationResult<T>> SeparateAsync(Tensor<T> audio, CancellationToken cancellationToken) |
Separates all sources asynchronously. |
Tensor<T> ExtractSource(Tensor<T> audio, string source) |
Extracts a specific source from the mix. |
Tensor<T> ForwardForTraining(Tensor<T> input) |
Training forward pass. |
Tensor<T> GetSourceMask(Tensor<T> audio, string source) |
Gets the soft mask for a specific source. |
Tensor<T> Remix(SourceSeparationResult<T> separationResult, IReadOnlyDictionary<string, double> sourceVolumes) |
Remixes the separated sources with custom volumes. |
Tensor<T> RemoveSource(Tensor<T> audio, string source) |
Removes a specific source from the mix. |
Fields
| Field | Summary |
|---|---|
string[] FiveStemSources |
Source names for 5-stem separation. |
string[] StandardSources |
Standard source names for 4-stem separation. |
string[] TwoStemSources |
Source names for 2-stem separation. |