VoiceCraftOptions

Options & Configuration · AiDotNet.TextToSpeech.CodecBased

Configuration for VoiceCraft (Peng et al. 2024).

Inheritance: ModelOptions → TtsModelOptions → CodecTtsOptions → VoiceCraftOptions

How It Works

Defaults follow the released 830M model. Table 9 says 12 attention heads, which cannot divide its 2048-wide hidden states; released 830M checkpoints use 16. Official830M selects that released configuration. The released 330M model is not Table 9’s 120M ablation: it is 1024-wide, 24 layers and 16 heads, exposed by Official330M.

Constructors

Constructor Summary
VoiceCraftOptions() Creates the paper’s main 830M configuration.
VoiceCraftOptions(VoiceCraftOptions other) Copies a configuration.

Properties

Property Summary
EnCodecOptions Codec { get; set; } The frozen 16 kHz, 50 Hz, four-by-2048 EnCodec trained by the authors on GigaSpeech.
Func<string, IReadOnlyList<string>> TextPhonemizer { get; set; } Optional externally precomputed IPA phones using the checkpoint’s inventory.
bool UseScaledAdam { get; set; } Gets or sets whether training uses the ScaledAdam optimizer.
double AudioEmbeddingDropout { get; set; } Gets or sets the dropout probability for the audio embedding.
double AudioPositionDropout { get; set; } Gets or sets the dropout probability for the audio position.
double DropoutRate { get; set; } The causal Transformer’s training dropout, forwarded to TransformerDropout.
double Temperature { get; set; } Gets or sets the logit temperature used during sampling.
double TextEmbeddingDropout { get; set; } Gets or sets the dropout probability for the text embedding.
double TextPositionDropout { get; set; } Gets or sets the dropout probability for the text position.
double TopP { get; set; } Gets or sets the cumulative probability threshold for nucleus sampling.
double TransformerDropout { get; set; } Gets or sets the dropout probability for the transformer.
double[] CodebookLossWeights { get; set; } Gets or sets the codebook loss weights.
int AudioPaddingToken { get; } Gets or sets the token ID for audio padding.
int AudioVocabularySize { get; } Gets or sets the audio vocabulary size.
int CodebookSize { get; set; } The codec vocabulary, paired with the nested codec configuration and bandwidth.
int CodecFrameRate { get; set; } The GigaSpeech codec’s fixed 50 Hz frame rate.
int EdenEpoch { get; set; } Gets or sets the eden epoch.
int EdenStep { get; set; } Gets or sets the eden step.
int EdenWarmupSteps { get; set; } Gets or sets the eden warmup steps.
int EmptyToken { get; } Gets or sets the token ID for empty.
int EndOfGenerationToken { get; } Gets or sets the token ID for end of generation.
int EndOfUtteranceToken { get; } Gets or sets the token ID for end of utterance.
int FeedForwardDim { get; set; } Gets or sets the hidden width of the feed-forward network.
int HiddenDim { get; set; } The causal Transformer width, forwarded to LLMDim.
int HopSize { get; set; } The GigaSpeech codec’s fixed 320-sample hop.
int MaxMaskSpans { get; set; } Gets or sets the max mask spans.
int MaximumFramesPerPhone { get; set; } Gets or sets the maximum frames per phone.
int MaximumMaskFrames { get; set; } Gets or sets the maximum mask frames.
int MinimumGapFrames { get; set; } Gets or sets the minimum gap frames.
int MinimumGeneratedFrames { get; set; } Gets or sets the minimum generated frames.
int MinimumMaskFrames { get; set; } Gets or sets the minimum mask frames.
int NumCodebooks { get; set; } The codebook count, paired with the codec bandwidth, quantizer count and loss weights.
int NumDecoderLayers { get; set; } The causal Transformer stack depth, forwarded to NumLLMLayers.
int NumEncoderLayers { get; set; } The causal Transformer stack depth, forwarded to NumLLMLayers.
int PseudoEpochSteps { get; set; } Gets or sets the pseudo epoch steps.
int SampleBatchSize { get; set; } Gets or sets the sample batch size.
int SampleRate { get; set; } The GigaSpeech codec’s fixed 16000 Hz sample rate.
int SamplingSeed { get; set; } Gets or sets the random seed used during sampling.
int StopSilenceRepetition { get; set; } Gets or sets the stop silence repetition.
int TextVocabularySize { get; set; } Gets or sets the text vocabulary size.
int TopK { get; set; } Top-k sampling.
int TotalTrainingSteps { get; set; } Gets or sets the total training steps.
int VocabSize { get; set; } The text embedding vocabulary, forwarded to TextVocabularySize.
int[] SilenceTokens { get; set; } Gets or sets the silence tokens.

Methods

Method Summary
VoiceCraftOptions Official330M() Released 330M layout.
VoiceCraftOptions Official330MTtsEnhanced() The authors’ TTS-enhanced 330M layout.
VoiceCraftOptions Official830M() Released 830M layout, including the release’s 16 heads and current top-40 sampling.