StableAudioModel<T>
Models & Types · AiDotNet.Diffusion.Audio
Stable Audio Open model — DiT-based latent diffusion for long-form audio generation.
Inheritance: DiffusionModelBase<T> → LatentDiffusionModelBase<T> → AudioDiffusionModelBase<T> → StableAudioModel<T>
For Beginners
Stable Audio creates music and sound effects from text:
How Stable Audio works:
- Text is encoded using a CLAP text encoder for audio-aware embeddings
- A DiT (Diffusion Transformer) denoises latents conditioned on text and timing
- Timing embeddings (start_s, total_s) control where and how long to generate
- An autoencoder decodes the latents back to 44.1 kHz stereo audio
Key characteristics:
- DiT backbone with timing conditioning (start_s, total_s)
- 44.1 kHz sample rate for high-fidelity audio
- Autoencoder-based latent space
- Up to 47 seconds of stereo audio
- Supports both music and sound effect generation
Advantages:
- High audio quality at 44.1 kHz
- Variable-length generation via timing conditioning
- Strong prompt adherence through CLAP embeddings
- Open-source weights and architecture
Limitations:
- Maximum duration of 47 seconds
- No text-to-speech capability
- Requires significant compute for real-time generation
Reference: Evans et al., “Stable Audio Open”, Stability AI, 2024
How It Works
Stable Audio uses a DiT operating on audio latents with timing conditioning for variable-length, high-quality audio generation at 44.1 kHz.
Example
var originalAudio = Tensor<float>.CreateRandom(1, 16000);
// Create a Stable Audio model with default settings
var stableAudio = new StableAudioModel<float>();
// Generate ambient music from a text prompt
var music = stableAudio.GenerateFromText(
prompt: "Ambient electronic music with soft pads and gentle arpeggios",
durationSeconds: 30.0,
numInferenceSteps: 100,
guidanceScale: 3.5,
seed: 42);
// Transform existing audio with a new style
var transformed = stableAudio.AudioToAudio(
inputAudio: originalAudio,
prompt: "Lo-fi version with vinyl crackle",
strength: 0.6);
Constructors
| Constructor | Summary |
|---|---|
StableAudioModel(NeuralNetworkArchitecture<T> architecture, DiffusionModelOptions<T> options, INoiseScheduler<T> scheduler, DiTNoisePredictor<T> dit, AudioVAE<T> audioVAE, IConditioningModule<T> conditioner) |
Initializes a new instance of StableAudioModel with full customization support. |
Properties
| Property | Summary |
|---|---|
IConditioningModule<T> Conditioner { get; } |
|
INoisePredictor<T> NoisePredictor { get; } |
|
IVAEModel<T> VAE { get; } |
|
bool SupportsAudioToAudio { get; } |
|
bool SupportsTextToAudio { get; } |
|
bool SupportsTextToMusic { get; } |
|
bool SupportsTextToSpeech { get; } |
|
int LatentChannels { get; } |
Methods
| Method | Summary |
|---|---|
ModelMetadata<T> GetModelMetadata() |