StableAudioModel<T>
Models & Types · AiDotNet.Audio.StableAudio
Stable Audio model for generating high-quality audio from text descriptions.
Inheritance: NeuralNetworkBase<T> → AudioNeuralNetworkBase<T> → StableAudioModel<T>
Implements: IAudioGenerator<T>
For Beginners
Stable Audio creates professional-quality audio:
How it works:
- You describe the audio you want (“upbeat electronic track”)
- T5 encodes your text into embeddings
- Duration and timing are encoded as conditioning
- DiT diffusion generates latent audio representations
- VAE decoder converts latents to 44.1kHz stereo audio
Key features:
- CD-quality 44.1kHz stereo output
- Variable-length generation (up to 3 minutes)
- Music and sound effects generation
- Timing-aware conditioning
Usage:
How It Works
Stable Audio is Stability AI’s state-of-the-art audio generation model that uses latent diffusion with a Diffusion Transformer (DiT) architecture for high-quality music and sound effects generation.
Architecture components:
- T5 Text Encoder: Encodes text prompts into conditioning embeddings
- VAE: Compresses audio to/from latent space (44.1kHz to 21.5Hz latent)
- DiT (Diffusion Transformer): Predicts noise using transformer blocks
- Timing Conditioning: Encodes duration and timing information
Reference: “Stable Audio: Fast Timing-Conditioned Latent Audio Diffusion” by Evans et al., 2024
Constructors
| Constructor | Summary |
|---|---|
StableAudioModel(NeuralNetworkArchitecture<T> architecture, StableAudioOptions options, ITokenizer tokenizer, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer, ILossFunction<T> lossFunction) |
Creates a Stable Audio model using native layers for training from scratch. |
StableAudioModel(NeuralNetworkArchitecture<T> architecture, string textEncoderPath, string vaePath, string ditPath, ITokenizer tokenizer, StableAudioOptions options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer, ILossFunction<T> lossFunction) |
Creates a Stable Audio model using pretrained ONNX models for inference. |
Properties
| Property | Summary |
|---|---|
bool SupportsAudioContinuation { get; } |
Gets whether this model supports audio continuation. |
bool SupportsAudioInpainting { get; } |
Gets whether this model supports audio inpainting. |
bool SupportsTextToAudio { get; } |
Gets whether this model supports text-to-audio generation. |
bool SupportsTextToMusic { get; } |
Gets whether this model supports text-to-music generation. |
double MaxDurationSeconds { get; } |
Gets the maximum duration of audio that can be generated. |
int SampleRate { get; } |
Gets the sample rate of generated audio. |
Methods
| Method | Summary |
|---|---|
AudioGenerationOptions<T> GetDefaultOptions() |
Gets default generation options. |
IReadOnlyList<OutputAxisContract> OutputAxesFor(int inputRank) |
|
ModelMetadata<T> GetModelMetadata() |
Gets model metadata. |
Task<Tensor<T>> GenerateAudioAsync(string prompt, string negativePrompt, double durationSeconds, int numInferenceSteps, double guidanceScale, int? seed, CancellationToken cancellationToken) |
Generates audio asynchronously. |
Tensor<T> ContinueAudio(Tensor<T> inputAudio, string prompt, double extensionSeconds, int numInferenceSteps, int? seed) |
Continues existing audio by extending it. |
Tensor<T> GenerateAudio(string prompt, string negativePrompt, double durationSeconds, int numInferenceSteps, double guidanceScale, int? seed) |
Generates audio from a text description. |
Tensor<T> GenerateMusic(string prompt, string negativePrompt, double durationSeconds, int numInferenceSteps, double guidanceScale, int? seed) |
Generates music from a text description. |
Tensor<T> InpaintAudio(Tensor<T> audio, Tensor<T> mask, string prompt, int numInferenceSteps, int? seed) |
Fills in missing or masked sections of audio. |