MARS5TTSOptions
Options & Configuration · AiDotNet.TextToSpeech.CodecBased
Options for MARS5TTS.
Inheritance: ModelOptions → TtsModelOptions → MARS5TTSOptions
For Beginners
MARS5 copies a voice from about six seconds of recording. Give it the recording’s transcript too (“deep clone”) for a closer copy.
How It Works
MARS5 (Camb.ai, 2024) has no paper; the release (Camb-ai/MARS5-TTS) defines the model and its inference. The
defaults are the release’s (inference.py’s model sizes and InferenceConfig). The release ships no training
code: the losses follow its architecture notes and its diffusion code, with every unstated choice an option here.
The text and speech tokenizers are stored in the released AR checkpoint’s metadata; build the options with
FromCheckpoint to load them. Without it, byte-level tokenizers with no merges stand in, so an untrained
model works end to end.
Constructors
| Constructor | Summary |
|---|---|
MARS5TTSOptions() |
Creates the release’s configuration. |
Properties
| Property | Summary |
|---|---|
EnCodecOptions Codec { get; set; } |
The codec (EnCodec 24 kHz at 6 kbps). |
Mars5Stage CurrentStage { get; set; } |
The network MARS5TTS’s training updates. |
bool DeepClone { get; set; } |
Whether to deep clone (continue the reference’s transcript and codes) when the voice has a transcript. |
bool IncludeKlPrior { get; set; } |
Whether the NAR loss adds the KL prior in place of the t = 0 decoder term (compute_Lt’s default). |
double ArBeta1 { get; set; } |
The AR’s AdamW β₁ (PyTorch’s 0.9; the AR config names none). |
double ArBeta2 { get; set; } |
The AR’s AdamW β₂ (PyTorch’s 0.999). |
double ArEndLearningRate { get; set; } |
The AR’s final learning rate (2e-5). |
double ArLearningRate { get; set; } |
The AR’s peak learning rate (the checkpoint’s 4e-4). |
double ArReferenceMaxSeconds { get; set; } |
The AR’s longest speaker reference in seconds (8). |
double ArReferenceMinSeconds { get; set; } |
The AR’s speaker reference length range in seconds, cropped from the recording when the sample has no separate reference (0.3 to 8, the checkpoint’s spk_ref_min/spk_ref_max). |
double ArWeightDecay { get; set; } |
The AR’s weight decay (0.1). |
double ConditionDropProbability { get; set; } |
The probability of dropping the NAR’s conditioning in training (p_cond_drop, 0.1). |
double DropoutRate { get; set; } |
The released MARS5 blocks use zero activation dropout. |
double EosEstimatedLengthFactor { get; set; } |
The expected generation length as a multiple of the text’s characters (1). |
double EosPenaltyDecay { get; set; } |
The end-of-speech penalty’s decay (0.5). |
double EosPenaltyFactor { get; set; } |
The end-of-speech penalty’s factor (1). |
double Epsilon { get; set; } |
AdamW’s ε (PyTorch’s 1e-8). |
double FrequencyPenalty { get; set; } |
The frequency penalty (3). |
double GradientClipNorm { get; set; } |
The gradient norm both stages clip to (1). |
double NarBeta1 { get; set; } |
The NAR’s AdamW β₁ (0.8). |
double NarBeta2 { get; set; } |
The NAR’s AdamW β₂ (0.99). |
double NarEndLearningRate { get; set; } |
The NAR’s final learning rate (5e-6). |
double NarGuidance { get; set; } |
The NAR’s classifier-free guidance weight (3). |
double NarLearningRate { get; set; } |
The NAR’s peak learning rate (1e-4). |
double NarPromptMaxSeconds { get; set; } |
The NAR’s longest prompt in seconds (14, prompt_length_max). |
double NarPromptMinSeconds { get; set; } |
The NAR’s shortest prompt in seconds (0.5, prompt_length_min). |
double NarWeightDecay { get; set; } |
The NAR’s weight decay (0.002). |
double PresencePenalty { get; set; } |
The presence penalty (0.4). |
double ReferencePadSeconds { get; set; } |
Silence prepended to the reference, in seconds (0). |
double Temperature { get; set; } |
The AR temperature (0.7). |
double TextLossWeight { get; set; } |
The AR loss’s weight on the text tokens’ cross-entropy (0.01, Tortoise’s and XTTS’s text weight; the release says only “a smaller weight”). |
double TopP { get; set; } |
The AR top-p (0.2). |
double TrimDb { get; set; } |
Silence trimmed from the output’s ends, in dB below its loudest frame (27). |
double TypicalP { get; set; } |
The AR typical-p mass (1: off). |
double X0Temperature { get; set; } |
The NAR’s x₀ temperature (0.7). |
int ArEndSteps { get; set; } |
The step the AR’s rate reaches its end (700,000). |
int ArFeedForward { get; set; } |
The AR decoder’s SwiGLU width (3584 = 1536 · 7/3). |
int ArHeadDim { get; set; } |
The AR decoder’s head width (64: Mistral’s default, which the release never overrides). |
int ArLayers { get; set; } |
The AR decoder’s layers (26); its width is HiddenDim (1536) and its heads NumHeads (24). |
int ArSpeakerLayers { get; set; } |
The AR speaker encoder’s layers (2). |
int CodebookSize { get; set; } |
Codes per codebook (1024; the NAR adds one pad class). |
int DecoderDim { get; set; } |
Vocos’s width (384, vocos-encodec-24khz). |
int DecoderIntermediateDim { get; set; } |
Vocos’s feed-forward width (1152). |
int DecoderLayers { get; set; } |
Vocos’s layers (8). |
int DiffusionSteps { get; set; } |
Diffusion steps (200). |
int HopSize { get; set; } |
The waveform hop, paired with the codec’s stride product. |
int JumpLength { get; set; } |
RePaint’s jump length (1: no resampling). |
int JumpSamples { get; set; } |
RePaint’s resamples per jump (1). |
int MaxGenerationLength { get; set; } |
The longest AR sequence, prompt included (2000). |
int NarDecoderLayers { get; set; } |
The NAR’s decoder layers (16). |
int NarDim { get; set; } |
The NAR’s width (1024). |
int NarEncoderLayers { get; set; } |
The NAR’s encoder layers (8). |
int NarEndSteps { get; set; } |
The step the NAR’s rate reaches its end (800,000). |
int NarHeads { get; set; } |
The NAR’s heads (16). |
int NarSpeakerLayers { get; set; } |
The NAR speaker encoder’s layers (3). |
int NumCodebooks { get; set; } |
The codebooks the models read (8). |
int NumEncoderLayers { get; set; } |
The NAR text encoder’s depth. |
int PenaltyWindow { get; set; } |
Tokens the penalties look back over (80, about 5 s). |
int Q0OverrideSteps { get; set; } |
The final diffusion steps at which the NAR may revise the first codebook (20). |
int SampleRate { get; set; } |
The waveform rate, paired with Codec.SampleRate. |
int SamplingSeed { get; set; } |
The seed of sampling and diffusion. |
int SlidingWindow { get; set; } |
The AR decoder’s attention window (3000). |
int TimeDim { get; set; } |
The timestep embedding’s width (1024). |
int TopK { get; set; } |
The AR top-k (200). |
int TrainingSeed { get; set; } |
The training draws’ seed. |
int VocabSize { get; set; } |
Inherited vocabulary metadata; TextTokenizerModel defines the native vocabulary. |
int VocosBandwidthId { get; set; } |
The Vocos bandwidth class decoding uses (1, the 3 kbps class: the release’s smoothing). |
int WarmupSteps { get; set; } |
Warmup steps of both rates (10,000), linear from zero; the rate then falls linearly to its end (the schedule’s shape is not recorded). |
string SpeechTokenizerModel { get; set; } |
The speech tokenizer’s minbpe model (speechtok.model); null for codes with no merges and the end-of-speech special. |
string TextTokenizerModel { get; set; } |
The text tokenizer’s minbpe model (the AR checkpoint’s texttok.model); null for a byte-level tokenizer with no merges and the two text specials. |
Methods
| Method | Summary |
|---|---|
EnCodecOptions CreateReleaseCodec() |
EnCodec’s 24 kHz model at 6 kbps (eight codebooks), as the release encodes with it. |
MARS5TTSOptions FromCheckpoint(string arCheckpointPath) |
The release’s configuration with the tokenizers stored in an AR checkpoint’s metadata (mars5_ar.safetensors). |