MaskGCTOptions
Options & Configuration · AiDotNet.TextToSpeech.FlowDiffusion
Options for MaskGCT.
Inheritance: ModelOptions → MaskGCTOptions
For Beginners
MaskGCT clones a voice from a short recording and its transcript: these settings size its networks and control how many refinement passes it makes; the defaults reproduce the released model.
How It Works
The defaults are the released MaskGCT (Wang et al. 2024; amphion/MaskGCT, Amphion
models/tts/maskgct/config/maskgct.json): semantic tokens from w2v-BERT 2.0’s 17th hidden layer, normalized and
quantized by a RepCodec of 8,192 codes; acoustic tokens from a 24 kHz codec of twelve 1,024-code layers at 50 Hz; a
text-to-semantic transformer of width 1536 and a pair of semantic-to-acoustic transformers of width 1024, all 16
layers with 16 heads. Decoding follows the paper where it is explicit (§4.1: 50 text-to-semantic steps,
semantic-to-acoustic steps 40, 16 and then 1 per layer, guidance 2.5 rescaled by 0.75) and the release otherwise
(temperatures 0.9 and 1.5, top-k keeping the best 2%).
Constructors
| Constructor | Summary |
|---|---|
MaskGCTOptions() |
Creates the released configuration. |
Properties
| Property | Summary |
|---|---|
MaskGctActivation Activation { get; set; } |
The generators’ gated activation: SiLU, as the released models were trained, or GELU, as the paper writes. |
MaskGctDurationMode DurationMode { get; set; } |
How the target’s length is chosen (the release’s rule by default). |
MaskGctStage CurrentStage { get; set; } |
The network training updates. |
OnnxModelOptions OnnxOptions { get; set; } |
Settings for the exported ONNX graph runtime. |
double Beta1 { get; set; } |
AdamW’s β₁ (0.9). |
double Beta2 { get; set; } |
AdamW’s β₂ (0.999). |
double DurationGuidance { get; set; } |
The duration predictor’s guidance toward the prompted velocity (1: none; the paper uses guidance but gives no scale). |
double DurationMultiplier { get; set; } |
A factor on the chosen duration (§4.2.3’s total-duration multiplier; 1 keeps it). |
double Epsilon { get; set; } |
AdamW’s ε (1e-8). |
double FilterThreshold { get; set; } |
The top-k filter: logits below the best (1 − this) of the vocabulary are dropped (0.98). |
double Guidance { get; set; } |
The classifier-free guidance scale (2.5; 0 turns guidance off). |
double GuidanceRescale { get; set; } |
The guidance rescale (0.75). |
double LearningRate { get; set; } |
The peak learning rate of the warmup/inverse-square-root schedule. |
double PromptDropProbability { get; set; } |
The probability a training example has no prompt (0.15, the release’s cfg_scale and App. |
double SemanticToAcousticTemperature { get; set; } |
The semantic-to-acoustic sampling temperature (1.5, the release’s). |
double TextToSemanticTemperature { get; set; } |
The text-to-semantic sampling temperature (0.9, the release’s). |
double WeightDecay { get; set; } |
The AdamW weight decay. |
double? TargetDurationSeconds { get; set; } |
The target’s duration in seconds, overriding DurationMode (the release’s target_len); null to choose it. |
int CodebookDim { get; set; } |
The acoustic codebooks’ dimension (8). |
int CodebookSize { get; set; } |
Each acoustic layer’s codes (1,024). |
int CodecEncoderDim { get; set; } |
The acoustic codec’s encoder width (96). |
int CodecLatentDim { get; set; } |
The acoustic codec’s latent width (256). |
int DurationDim { get; set; } |
The duration predictor’s width (768, App. |
int DurationHeads { get; set; } |
The duration predictor’s attention heads (12). |
int DurationLayers { get; set; } |
The duration predictor’s layers (12). |
int DurationSteps { get; set; } |
The midpoint ODE solver’s steps (4). |
int HiddenDim { get; set; } |
The semantic-to-acoustic transformer width. |
int HopSize { get; set; } |
Samples per acoustic frame; must match the codec encoder strides. |
int MaxTextLength { get; set; } |
The maximum input phone sequence length. |
int NumQuantizers { get; set; } |
The acoustic layers (12). |
int SampleRate { get; set; } |
The acoustic codec’s waveform sample rate (24 kHz). |
int SamplingSeed { get; set; } |
The seed of decoding’s random draws. |
int SemanticCodebookDim { get; set; } |
The semantic codebook’s dimension (8). |
int SemanticCodebookSize { get; set; } |
The semantic codebook’s size (8,192). |
int SemanticCodecDim { get; set; } |
The semantic codec’s Vocos width (384). |
int SemanticCodecIntermediate { get; set; } |
The semantic codec’s Vocos feed-forward width (2048). |
int SemanticCodecLayers { get; set; } |
The semantic codec’s ConvNeXt blocks (12). |
int SemanticHeads { get; set; } |
w2v-BERT 2.0’s attention heads. |
int SemanticHiddenSize { get; set; } |
w2v-BERT 2.0’s width. |
int SemanticIntermediateSize { get; set; } |
w2v-BERT 2.0’s feed-forward width. |
int SemanticLayer { get; set; } |
The w2v-BERT 2.0 hidden state the semantic codec reads (17; only that many layers are built). |
int SemanticSampleRate { get; set; } |
The rate the semantic model reads (16 kHz). |
int SemanticToAcousticHeads { get; set; } |
The semantic-to-acoustic models’ attention heads (16). |
int SemanticToAcousticLayers { get; set; } |
The semantic-to-acoustic models’ layers (16; their width is HiddenDim). |
int TextToSemanticDim { get; set; } |
The text-to-semantic model’s width (1536). |
int TextToSemanticHeads { get; set; } |
The text-to-semantic model’s attention heads (16). |
int TextToSemanticLayers { get; set; } |
The text-to-semantic model’s layers (16). |
int TextToSemanticSteps { get; set; } |
Text-to-semantic decoding steps (50, §4.1; the release’s pipeline defaults to 25). |
int TrainingSeed { get; set; } |
The seed of training’s random choices (the step, the prompt, the masks). |
int VocabSize { get; set; } |
The phone vocabulary used by the text and duration models. |
int VocosDim { get; set; } |
The acoustic decoder’s Vocos width (512). |
int VocosFftSize { get; set; } |
The acoustic decoder’s ISTFT size (1920). |
int VocosIntermediate { get; set; } |
The acoustic decoder’s Vocos feed-forward width (4096). |
int VocosLayers { get; set; } |
The acoustic decoder’s ConvNeXt blocks (30). |
int WarmupSteps { get; set; } |
The learning rate’s warmup steps (32,000, §4.1), after which it decays with the inverse square root of the step. |
int[] CodecEncoderRates { get; set; } |
The acoustic codec’s encoder strides (3, 4, 5, 8: 480 samples a frame). |
int[] SemanticToAcousticSteps { get; set; } |
Semantic-to-acoustic decoding steps per layer (40, 16, then 1, §4.1; the release’s pipeline uses 25, 10, then 1). |
string Language { get; set; } |
The language of the text to synthesize: “en”, “zh”, “ja”, “ko”, “fr” or “de”. |
string ModelPath { get; set; } |
The exported ONNX graph path, when a graph is loaded. |
string OpenJTalkDictionaryPath { get; set; } |
The Open JTalk dictionary directory (open_jtalk_dic_utf_8-1.11, as pyopenjtalk downloads it), which Japanese text needs. |
string PolyphoneModelPath { get; set; } |
The release’s Chinese polyphone model (g2p/sources/g2p_chinese_model/poly_bert_model.onnx), which Mandarin text needs. |