MaskGCTOptions

Options & Configuration · AiDotNet.TextToSpeech.FlowDiffusion

Options for MaskGCT.

Inheritance: ModelOptions → MaskGCTOptions

For Beginners

MaskGCT clones a voice from a short recording and its transcript: these settings size its networks and control how many refinement passes it makes; the defaults reproduce the released model.

How It Works

The defaults are the released MaskGCT (Wang et al. 2024; amphion/MaskGCT, Amphion models/tts/maskgct/config/maskgct.json): semantic tokens from w2v-BERT 2.0’s 17th hidden layer, normalized and quantized by a RepCodec of 8,192 codes; acoustic tokens from a 24 kHz codec of twelve 1,024-code layers at 50 Hz; a text-to-semantic transformer of width 1536 and a pair of semantic-to-acoustic transformers of width 1024, all 16 layers with 16 heads. Decoding follows the paper where it is explicit (§4.1: 50 text-to-semantic steps, semantic-to-acoustic steps 40, 16 and then 1 per layer, guidance 2.5 rescaled by 0.75) and the release otherwise (temperatures 0.9 and 1.5, top-k keeping the best 2%).

Constructors

Constructor Summary
MaskGCTOptions() Creates the released configuration.

Properties

Property Summary
MaskGctActivation Activation { get; set; } The generators’ gated activation: SiLU, as the released models were trained, or GELU, as the paper writes.
MaskGctDurationMode DurationMode { get; set; } How the target’s length is chosen (the release’s rule by default).
MaskGctStage CurrentStage { get; set; } The network training updates.
OnnxModelOptions OnnxOptions { get; set; } Settings for the exported ONNX graph runtime.
double Beta1 { get; set; } AdamW’s β₁ (0.9).
double Beta2 { get; set; } AdamW’s β₂ (0.999).
double DurationGuidance { get; set; } The duration predictor’s guidance toward the prompted velocity (1: none; the paper uses guidance but gives no scale).
double DurationMultiplier { get; set; } A factor on the chosen duration (§4.2.3’s total-duration multiplier; 1 keeps it).
double Epsilon { get; set; } AdamW’s ε (1e-8).
double FilterThreshold { get; set; } The top-k filter: logits below the best (1 − this) of the vocabulary are dropped (0.98).
double Guidance { get; set; } The classifier-free guidance scale (2.5; 0 turns guidance off).
double GuidanceRescale { get; set; } The guidance rescale (0.75).
double LearningRate { get; set; } The peak learning rate of the warmup/inverse-square-root schedule.
double PromptDropProbability { get; set; } The probability a training example has no prompt (0.15, the release’s cfg_scale and App.
double SemanticToAcousticTemperature { get; set; } The semantic-to-acoustic sampling temperature (1.5, the release’s).
double TextToSemanticTemperature { get; set; } The text-to-semantic sampling temperature (0.9, the release’s).
double WeightDecay { get; set; } The AdamW weight decay.
double? TargetDurationSeconds { get; set; } The target’s duration in seconds, overriding DurationMode (the release’s target_len); null to choose it.
int CodebookDim { get; set; } The acoustic codebooks’ dimension (8).
int CodebookSize { get; set; } Each acoustic layer’s codes (1,024).
int CodecEncoderDim { get; set; } The acoustic codec’s encoder width (96).
int CodecLatentDim { get; set; } The acoustic codec’s latent width (256).
int DurationDim { get; set; } The duration predictor’s width (768, App.
int DurationHeads { get; set; } The duration predictor’s attention heads (12).
int DurationLayers { get; set; } The duration predictor’s layers (12).
int DurationSteps { get; set; } The midpoint ODE solver’s steps (4).
int HiddenDim { get; set; } The semantic-to-acoustic transformer width.
int HopSize { get; set; } Samples per acoustic frame; must match the codec encoder strides.
int MaxTextLength { get; set; } The maximum input phone sequence length.
int NumQuantizers { get; set; } The acoustic layers (12).
int SampleRate { get; set; } The acoustic codec’s waveform sample rate (24 kHz).
int SamplingSeed { get; set; } The seed of decoding’s random draws.
int SemanticCodebookDim { get; set; } The semantic codebook’s dimension (8).
int SemanticCodebookSize { get; set; } The semantic codebook’s size (8,192).
int SemanticCodecDim { get; set; } The semantic codec’s Vocos width (384).
int SemanticCodecIntermediate { get; set; } The semantic codec’s Vocos feed-forward width (2048).
int SemanticCodecLayers { get; set; } The semantic codec’s ConvNeXt blocks (12).
int SemanticHeads { get; set; } w2v-BERT 2.0’s attention heads.
int SemanticHiddenSize { get; set; } w2v-BERT 2.0’s width.
int SemanticIntermediateSize { get; set; } w2v-BERT 2.0’s feed-forward width.
int SemanticLayer { get; set; } The w2v-BERT 2.0 hidden state the semantic codec reads (17; only that many layers are built).
int SemanticSampleRate { get; set; } The rate the semantic model reads (16 kHz).
int SemanticToAcousticHeads { get; set; } The semantic-to-acoustic models’ attention heads (16).
int SemanticToAcousticLayers { get; set; } The semantic-to-acoustic models’ layers (16; their width is HiddenDim).
int TextToSemanticDim { get; set; } The text-to-semantic model’s width (1536).
int TextToSemanticHeads { get; set; } The text-to-semantic model’s attention heads (16).
int TextToSemanticLayers { get; set; } The text-to-semantic model’s layers (16).
int TextToSemanticSteps { get; set; } Text-to-semantic decoding steps (50, §4.1; the release’s pipeline defaults to 25).
int TrainingSeed { get; set; } The seed of training’s random choices (the step, the prompt, the masks).
int VocabSize { get; set; } The phone vocabulary used by the text and duration models.
int VocosDim { get; set; } The acoustic decoder’s Vocos width (512).
int VocosFftSize { get; set; } The acoustic decoder’s ISTFT size (1920).
int VocosIntermediate { get; set; } The acoustic decoder’s Vocos feed-forward width (4096).
int VocosLayers { get; set; } The acoustic decoder’s ConvNeXt blocks (30).
int WarmupSteps { get; set; } The learning rate’s warmup steps (32,000, §4.1), after which it decays with the inverse square root of the step.
int[] CodecEncoderRates { get; set; } The acoustic codec’s encoder strides (3, 4, 5, 8: 480 samples a frame).
int[] SemanticToAcousticSteps { get; set; } Semantic-to-acoustic decoding steps per layer (40, 16, then 1, §4.1; the release’s pipeline uses 25, 10, then 1).
string Language { get; set; } The language of the text to synthesize: “en”, “zh”, “ja”, “ko”, “fr” or “de”.
string ModelPath { get; set; } The exported ONNX graph path, when a graph is loaded.
string OpenJTalkDictionaryPath { get; set; } The Open JTalk dictionary directory (open_jtalk_dic_utf_8-1.11, as pyopenjtalk downloads it), which Japanese text needs.
string PolyphoneModelPath { get; set; } The release’s Chinese polyphone model (g2p/sources/g2p_chinese_model/poly_bert_model.onnx), which Mandarin text needs.