VoiceCraftOptions
Options & Configuration · AiDotNet.TextToSpeech.CodecBased
Configuration for VoiceCraft (Peng et al. 2024).
Inheritance: ModelOptions → TtsModelOptions → CodecTtsOptions → VoiceCraftOptions
How It Works
Defaults follow the released 830M model. Table 9 says 12 attention heads, which cannot divide its 2048-wide
hidden states; released 830M checkpoints use 16. Official830M selects that released configuration.
The released 330M model is not
Table 9’s 120M ablation: it is 1024-wide, 24 layers and 16 heads, exposed by Official330M.
Constructors
| Constructor | Summary |
|---|---|
VoiceCraftOptions() |
Creates the paper’s main 830M configuration. |
VoiceCraftOptions(VoiceCraftOptions other) |
Copies a configuration. |
Properties
| Property | Summary |
|---|---|
EnCodecOptions Codec { get; set; } |
The frozen 16 kHz, 50 Hz, four-by-2048 EnCodec trained by the authors on GigaSpeech. |
Func<string, IReadOnlyList<string>> TextPhonemizer { get; set; } |
Optional externally precomputed IPA phones using the checkpoint’s inventory. |
bool UseScaledAdam { get; set; } |
Gets or sets whether training uses the ScaledAdam optimizer. |
double AudioEmbeddingDropout { get; set; } |
Gets or sets the dropout probability for the audio embedding. |
double AudioPositionDropout { get; set; } |
Gets or sets the dropout probability for the audio position. |
double DropoutRate { get; set; } |
The causal Transformer’s training dropout, forwarded to TransformerDropout. |
double Temperature { get; set; } |
Gets or sets the logit temperature used during sampling. |
double TextEmbeddingDropout { get; set; } |
Gets or sets the dropout probability for the text embedding. |
double TextPositionDropout { get; set; } |
Gets or sets the dropout probability for the text position. |
double TopP { get; set; } |
Gets or sets the cumulative probability threshold for nucleus sampling. |
double TransformerDropout { get; set; } |
Gets or sets the dropout probability for the transformer. |
double[] CodebookLossWeights { get; set; } |
Gets or sets the codebook loss weights. |
int AudioPaddingToken { get; } |
Gets or sets the token ID for audio padding. |
int AudioVocabularySize { get; } |
Gets or sets the audio vocabulary size. |
int CodebookSize { get; set; } |
The codec vocabulary, paired with the nested codec configuration and bandwidth. |
int CodecFrameRate { get; set; } |
The GigaSpeech codec’s fixed 50 Hz frame rate. |
int EdenEpoch { get; set; } |
Gets or sets the eden epoch. |
int EdenStep { get; set; } |
Gets or sets the eden step. |
int EdenWarmupSteps { get; set; } |
Gets or sets the eden warmup steps. |
int EmptyToken { get; } |
Gets or sets the token ID for empty. |
int EndOfGenerationToken { get; } |
Gets or sets the token ID for end of generation. |
int EndOfUtteranceToken { get; } |
Gets or sets the token ID for end of utterance. |
int FeedForwardDim { get; set; } |
Gets or sets the hidden width of the feed-forward network. |
int HiddenDim { get; set; } |
The causal Transformer width, forwarded to LLMDim. |
int HopSize { get; set; } |
The GigaSpeech codec’s fixed 320-sample hop. |
int MaxMaskSpans { get; set; } |
Gets or sets the max mask spans. |
int MaximumFramesPerPhone { get; set; } |
Gets or sets the maximum frames per phone. |
int MaximumMaskFrames { get; set; } |
Gets or sets the maximum mask frames. |
int MinimumGapFrames { get; set; } |
Gets or sets the minimum gap frames. |
int MinimumGeneratedFrames { get; set; } |
Gets or sets the minimum generated frames. |
int MinimumMaskFrames { get; set; } |
Gets or sets the minimum mask frames. |
int NumCodebooks { get; set; } |
The codebook count, paired with the codec bandwidth, quantizer count and loss weights. |
int NumDecoderLayers { get; set; } |
The causal Transformer stack depth, forwarded to NumLLMLayers. |
int NumEncoderLayers { get; set; } |
The causal Transformer stack depth, forwarded to NumLLMLayers. |
int PseudoEpochSteps { get; set; } |
Gets or sets the pseudo epoch steps. |
int SampleBatchSize { get; set; } |
Gets or sets the sample batch size. |
int SampleRate { get; set; } |
The GigaSpeech codec’s fixed 16000 Hz sample rate. |
int SamplingSeed { get; set; } |
Gets or sets the random seed used during sampling. |
int StopSilenceRepetition { get; set; } |
Gets or sets the stop silence repetition. |
int TextVocabularySize { get; set; } |
Gets or sets the text vocabulary size. |
int TopK { get; set; } |
Top-k sampling. |
int TotalTrainingSteps { get; set; } |
Gets or sets the total training steps. |
int VocabSize { get; set; } |
The text embedding vocabulary, forwarded to TextVocabularySize. |
int[] SilenceTokens { get; set; } |
Gets or sets the silence tokens. |
Methods
| Method | Summary |
|---|---|
VoiceCraftOptions Official330M() |
Released 330M layout. |
VoiceCraftOptions Official330MTtsEnhanced() |
The authors’ TTS-enhanced 330M layout. |
VoiceCraftOptions Official830M() |
Released 830M layout, including the release’s 16 heads and current top-40 sampling. |