VoiceboxOptions

Options & Configuration · AiDotNet.TextToSpeech.CodecBased

Configuration of the English Voicebox audio and duration models.

Inheritance: ModelOptions → TtsModelOptions → VoiceboxOptions

How It Works

Defaults reproduce Le et al. (2023), Appendix D, Tables D13-D14 and Appendix A: a 24-layer, 1024-wide audio Transformer; an 8-layer, 512-wide regression duration Transformer; normalized 80-bin log-mel features at 100 Hz; midpoint integration with 32 vector-field evaluations; and the 16 kHz adaptation of HiFi-GAN V1. The paper did not publish weights, source code, the phone table, or several ordinary Transformer implementation details; the corresponding choices are explicit options so converted, independently trained checkpoints cannot be mistaken for Meta checkpoints.

Constructors

Constructor Summary
VoiceboxOptions() Creates the paper’s English Voicebox configuration.
VoiceboxOptions(VoiceboxOptions other) Copies a Voicebox configuration.

Properties

Property Summary
Dictionary<string, int> PhoneVocabulary { get; set; } The dataset/checkpoint’s position-dependent MFA phone IDs, including SIL.
bool UseNativeEnglishFallback { get; set; } Explicitly enables the repository’s English ARPAbet frontend for independently trained compatible models; it is not the paper’s MFA frontend.
double AudioActivationDropout { get; set; } Audio feed-forward activation dropout (0.1).
double AudioAttentionDropout { get; set; } Audio attention dropout (0).
double ConditionalDropout { get; set; } Probability of dropping all conditions during audio or duration training (0.2).
double DurationActivationDropout { get; set; } Duration feed-forward activation dropout (0.1).
double DurationAttentionDropout { get; set; } Duration attention dropout (0.1).
double DurationDropProbability { get; set; } Probability of masking the complete duration sequence (0.2).
double DurationMaskMaximum { get; set; } Largest otherwise sampled duration mask fraction (1.0).
double DurationMaskMinimum { get; set; } Smallest otherwise sampled duration mask fraction (0.1).
double FlowTimeScale { get; set; } Scale applied before sinusoidal encoding of the unit-interval flow time.
double GradientClipNorm { get; set; } Audio-model gradient norm clip (0.2).
double GuidanceStrength { get; set; } English zero-shot classifier-free guidance strength (0.7).
double MelMaxFrequency { get; set; } Highest mel frequency (8 kHz).
double MelMean { get; set; } Global log-mel mean from Appendix A.3.
double MelMinFrequency { get; set; } Lowest mel frequency (0 Hz).
double MelStandardDeviation { get; set; } Global log-mel standard deviation from Appendix A.3.
double SigmaMinimum { get; set; } Terminal standard deviation of the optimal-transport path (1e-5).
double SpectrogramDropProbability { get; set; } Probability of masking the complete spectrogram (0.3).
double SpectrogramMaskMaximum { get; set; } Largest otherwise sampled spectrogram mask fraction (1.0).
double SpectrogramMaskMinimum { get; set; } Smallest otherwise sampled spectrogram mask fraction (0.7).
int AudioConvPositionDepth { get; set; } Audio convolutional-position depth (2).
int AudioConvPositionGroups { get; set; } Audio convolutional-position groups (16).
int AudioConvPositionWidth { get; set; } Audio convolutional-position kernel width (31).
int AudioFeedForwardDim { get; set; } Audio Transformer feed-forward width (4096).
int AudioHeads { get; set; } Audio Transformer attention heads (16).
int AudioLayers { get; set; } Audio Transformer layers (24).
int AudioModelDim { get; set; } Audio vector-field Transformer width (1024).
int CodebookSize { get; set; } Compatibility-only remnant of the former placeholder; Voicebox has no codec vocabulary.
int DurationConvPositionDepth { get; set; } Duration convolutional-position depth (2).
int DurationConvPositionGroups { get; set; } Duration convolutional-position groups (16).
int DurationConvPositionWidth { get; set; } Duration convolutional-position kernel width (15).
int DurationFeedForwardDim { get; set; } Duration Transformer feed-forward width (2048).
int DurationHeads { get; set; } Duration Transformer heads (8).
int DurationLayers { get; set; } English duration Transformer layers (8).
int DurationModelDim { get; set; } English regression-duration Transformer width (512).
int LLMDim { get; set; } Alias of AudioModelDim.
int MaxCodecFrames { get; set; } Compatibility alias for the continuous mel-frame limit.
int MaximumAudioFrames { get; set; } Training chunk limit in mel frames (1600).
int MaximumPhoneDuration { get; set; } Safety bound on an inferred phone duration; not specified by the paper.
int NumCodebooks { get; set; } Compatibility-only remnant of the former placeholder; Voicebox has no codec codebooks.
int NumHeads { get; set; } Alias of AudioHeads.
int NumLLMLayers { get; set; } Alias of AudioLayers.
int OdeSteps { get; set; } Midpoint integration steps (16, therefore 32 vector-field evaluations).
int PhoneEmbeddingDim { get; set; } Phone embedding width H; the paper does not publish H.
int PhoneVocabularySize { get; set; } Capacity of the converted position-dependent phone table.
int SamplingSeed { get; set; } Seed for native sampling and compact reproducibility tests.
int TextEncoderDim { get; set; } Alias of PhoneEmbeddingDim.
int TrainingUpdates { get; set; } English audio-model updates (500,000).
int VocoderInitialChannels { get; set; } HiFi-GAN V1 initial generator width (512).
int WarmupSteps { get; set; } Linear learning-rate warmup updates (5000).
int WindowLength { get; set; } Analysis window in samples (640, or 40 ms).
int[] VocoderResBlockKernels { get; set; } HiFi-GAN V1 multi-receptive-field kernels.
int[] VocoderUpsampleKernels { get; set; } Voicebox’s adjusted HiFi-GAN transposed-convolution kernels.
int[] VocoderUpsampleRates { get; set; } Voicebox’s 160x HiFi-GAN upsampling factors.
int[][] VocoderResBlockDilations { get; set; } HiFi-GAN V1 type-1 residual dilations.