StepAudioOptions

Options & Configuration · AiDotNet.TextToSpeech.MultiModal

Options for StepAudio.

Inheritance: ModelOptions → TtsModelOptions → StepAudioOptions

For Beginners

Step-Audio hears speech as two kinds of tokens (what is said and how it sounds), answers in text with a large language model, and speaks the answer with a smaller one.

How It Works

The defaults are the release (stepfun-ai Step-Audio-Chat, Step-Audio-TTS-3B, Step-Audio-Tokenizer) as the paper describes it: the dual-codebook tokenizer — the streaming Paraformer-large encoder’s 16.7 Hz frames quantized to 1,024 k-means codes, and CosyVoice’s S3 tokenizer at 25 Hz with 4,096 codes, interleaved 2:3 — the 130B Step-1 chat model, and the 3B Step-1 TTS model whose dual-codebook tokens CosyVoice-300M-25Hz’s flow and HiFT turn into 22.05 kHz speech, with a separate music decoder for RAP and humming.

Constructors

Constructor Summary
StepAudioOptions() Creates the release’s Step-Audio.

Properties

Property Summary
StepAudioTrainingStage Stage { get; set; } The stage a training step follows.
bool IncludeMusicDecoder { get; set; } Whether to build the music decoder (CosyVoice-300M-25Hz-Music) the release uses for RAP and humming.
bool IncludeRlhfModels { get; set; } Whether to build what RLHF needs (§5.2.3–5.2.6): a reward model and a critic (Step-1 chat-sized backbones with a scalar head) and the frozen reference policy for the KL penalty.
double ChatTemperature { get; set; } The chat model’s sampling (stepaudio.py: temperature 0.7, top-p 0.9, at most 2048 new tokens).
double ChatTopP { get; set; }
double Gamma { get; set; }
double KlCoefficient { get; set; }
double Lambda { get; set; }
double LinguisticDither { get; set; } The filterbanks’ Kaldi dither (the release’s 1.0; 0 makes the tokens deterministic).
double MaxGradientNorm { get; set; } Gradient clipping (not stated: 1.0).
double PpoClip { get; set; } PPO: the clip (0.2), the KL penalty (0.05), the critic’s warmup steps (80), GAE’s γ and λ (not stated: 1 and 0.95), and the sampled response length (not stated: 256).
double RepetitionRatio { get; set; }
double RmsEpsilon { get; set; } Step-1’s RMSNorm epsilon (1e-5).
double TtsTemperature { get; set; } The TTS model’s sampling (tts.py: temperature 0.7, the repetition-aware processor’s window 10 and ratio 0.1, at most 8192 tokens in all).
int ChatGroups { get; set; } The chat model’s key-value groups (8), layers (88) and feed-forward width (31,232); HiddenDim and NumHeads are its width (12,288) and heads (96).
int ChatIntermediate { get; set; }
int ChatLayers { get; set; }
int ChatMaxNewTokens { get; set; }
int CriticWarmupSteps { get; set; }
int DecoderBlocks { get; set; }
int DecoderHeadDim { get; set; }
int DecoderHeads { get; set; }
int DecoderMidBlocks { get; set; }
int FlowEncoderLayers { get; set; }
int FlowFeedForward { get; set; }
int FlowHeads { get; set; }
int FlowTimesteps { get; set; }
int FlowWidth { get; set; } The flow: width (512), heads (8), feed-forward (2048), encoder blocks (6), decoder channels (256, 256), decoder heads (8) of 64, blocks per level (4), middle blocks (12), Euler steps (10), mel bands (80).
int HiftBaseChannels { get; set; } HiFT: base channels (512), harmonics (8), upsampling (8, 8) with kernels (16, 16), source kernels (7, 11).
int HiftHarmonics { get; set; }
int LinguisticCodes { get; set; }
int MelChannelCount { get; set; }
int NumEncoderLayers { get; set; } Gets or sets the linguistic tokenizer’s Paraformer encoder block count.
int ParaformerBlocks { get; set; }
int ParaformerFeedForward { get; set; }
int ParaformerHeads { get; set; }
int ParaformerMels { get; set; } The linguistic tokenizer’s Paraformer encoder: filterbank bands (80), width (512), heads (4), feed-forward (2048), blocks (50); and its k-means codes (1,024).
int ParaformerWidth { get; set; }
int PpoResponseLength { get; set; }
int RepetitionWindow { get; set; }
int SemanticCodes { get; set; }
int SemanticHeads { get; set; }
int SemanticLayers { get; set; }
int SemanticWidth { get; set; } The semantic tokenizer (CosyVoice’s speech_tokenizer_v1): width (1280), heads (20), layers (6), codes (4,096).
int SpeakerEmbeddingDimension { get; set; } CAM++’s x-vector width (192), initial channels (128) and growth (32).
int SpeakerEncoderChannels { get; set; }
int SpeakerEncoderGrowthRate { get; set; }
int TotalSteps { get; set; } Steps of the stage’s cosine schedule (not stated; 10,000) and its warmup (not stated; 0).
int TtsGroups { get; set; }
int TtsHeads { get; set; }
int TtsIntermediate { get; set; }
int TtsLayers { get; set; }
int TtsMaxLength { get; set; }
int TtsMaxNewTokens { get; set; } A cap on the TTS model’s new tokens on top of TtsMaxLength (the release has none).
int TtsWidth { get; set; } The TTS model: width (3072), heads (48), key-value groups (4), layers (32), feed-forward (8192).
int WarmupSteps { get; set; }
int[] DecoderChannels { get; set; }
int[] HiftSourceKernels { get; set; }
int[] HiftUpsampleKernels { get; set; }
int[] HiftUpsampleRates { get; set; }
string TokenizerPath { get; set; } Step-1’s tokenizer (tokenizer.model); text needs it.