MARS5TTSOptions

Options & Configuration · AiDotNet.TextToSpeech.CodecBased

Options for MARS5TTS.

Inheritance: ModelOptions → TtsModelOptions → MARS5TTSOptions

For Beginners

MARS5 copies a voice from about six seconds of recording. Give it the recording’s transcript too (“deep clone”) for a closer copy.

How It Works

MARS5 (Camb.ai, 2024) has no paper; the release (Camb-ai/MARS5-TTS) defines the model and its inference. The defaults are the release’s (inference.py’s model sizes and InferenceConfig). The release ships no training code: the losses follow its architecture notes and its diffusion code, with every unstated choice an option here.

The text and speech tokenizers are stored in the released AR checkpoint’s metadata; build the options with FromCheckpoint to load them. Without it, byte-level tokenizers with no merges stand in, so an untrained model works end to end.

Constructors

Constructor Summary
MARS5TTSOptions() Creates the release’s configuration.

Properties

Property Summary
EnCodecOptions Codec { get; set; } The codec (EnCodec 24 kHz at 6 kbps).
Mars5Stage CurrentStage { get; set; } The network MARS5TTS’s training updates.
bool DeepClone { get; set; } Whether to deep clone (continue the reference’s transcript and codes) when the voice has a transcript.
bool IncludeKlPrior { get; set; } Whether the NAR loss adds the KL prior in place of the t = 0 decoder term (compute_Lt’s default).
double ArBeta1 { get; set; } The AR’s AdamW β₁ (PyTorch’s 0.9; the AR config names none).
double ArBeta2 { get; set; } The AR’s AdamW β₂ (PyTorch’s 0.999).
double ArEndLearningRate { get; set; } The AR’s final learning rate (2e-5).
double ArLearningRate { get; set; } The AR’s peak learning rate (the checkpoint’s 4e-4).
double ArReferenceMaxSeconds { get; set; } The AR’s longest speaker reference in seconds (8).
double ArReferenceMinSeconds { get; set; } The AR’s speaker reference length range in seconds, cropped from the recording when the sample has no separate reference (0.3 to 8, the checkpoint’s spk_ref_min/spk_ref_max).
double ArWeightDecay { get; set; } The AR’s weight decay (0.1).
double ConditionDropProbability { get; set; } The probability of dropping the NAR’s conditioning in training (p_cond_drop, 0.1).
double DropoutRate { get; set; } The released MARS5 blocks use zero activation dropout.
double EosEstimatedLengthFactor { get; set; } The expected generation length as a multiple of the text’s characters (1).
double EosPenaltyDecay { get; set; } The end-of-speech penalty’s decay (0.5).
double EosPenaltyFactor { get; set; } The end-of-speech penalty’s factor (1).
double Epsilon { get; set; } AdamW’s ε (PyTorch’s 1e-8).
double FrequencyPenalty { get; set; } The frequency penalty (3).
double GradientClipNorm { get; set; } The gradient norm both stages clip to (1).
double NarBeta1 { get; set; } The NAR’s AdamW β₁ (0.8).
double NarBeta2 { get; set; } The NAR’s AdamW β₂ (0.99).
double NarEndLearningRate { get; set; } The NAR’s final learning rate (5e-6).
double NarGuidance { get; set; } The NAR’s classifier-free guidance weight (3).
double NarLearningRate { get; set; } The NAR’s peak learning rate (1e-4).
double NarPromptMaxSeconds { get; set; } The NAR’s longest prompt in seconds (14, prompt_length_max).
double NarPromptMinSeconds { get; set; } The NAR’s shortest prompt in seconds (0.5, prompt_length_min).
double NarWeightDecay { get; set; } The NAR’s weight decay (0.002).
double PresencePenalty { get; set; } The presence penalty (0.4).
double ReferencePadSeconds { get; set; } Silence prepended to the reference, in seconds (0).
double Temperature { get; set; } The AR temperature (0.7).
double TextLossWeight { get; set; } The AR loss’s weight on the text tokens’ cross-entropy (0.01, Tortoise’s and XTTS’s text weight; the release says only “a smaller weight”).
double TopP { get; set; } The AR top-p (0.2).
double TrimDb { get; set; } Silence trimmed from the output’s ends, in dB below its loudest frame (27).
double TypicalP { get; set; } The AR typical-p mass (1: off).
double X0Temperature { get; set; } The NAR’s x₀ temperature (0.7).
int ArEndSteps { get; set; } The step the AR’s rate reaches its end (700,000).
int ArFeedForward { get; set; } The AR decoder’s SwiGLU width (3584 = 1536 · 7/3).
int ArHeadDim { get; set; } The AR decoder’s head width (64: Mistral’s default, which the release never overrides).
int ArLayers { get; set; } The AR decoder’s layers (26); its width is HiddenDim (1536) and its heads NumHeads (24).
int ArSpeakerLayers { get; set; } The AR speaker encoder’s layers (2).
int CodebookSize { get; set; } Codes per codebook (1024; the NAR adds one pad class).
int DecoderDim { get; set; } Vocos’s width (384, vocos-encodec-24khz).
int DecoderIntermediateDim { get; set; } Vocos’s feed-forward width (1152).
int DecoderLayers { get; set; } Vocos’s layers (8).
int DiffusionSteps { get; set; } Diffusion steps (200).
int HopSize { get; set; } The waveform hop, paired with the codec’s stride product.
int JumpLength { get; set; } RePaint’s jump length (1: no resampling).
int JumpSamples { get; set; } RePaint’s resamples per jump (1).
int MaxGenerationLength { get; set; } The longest AR sequence, prompt included (2000).
int NarDecoderLayers { get; set; } The NAR’s decoder layers (16).
int NarDim { get; set; } The NAR’s width (1024).
int NarEncoderLayers { get; set; } The NAR’s encoder layers (8).
int NarEndSteps { get; set; } The step the NAR’s rate reaches its end (800,000).
int NarHeads { get; set; } The NAR’s heads (16).
int NarSpeakerLayers { get; set; } The NAR speaker encoder’s layers (3).
int NumCodebooks { get; set; } The codebooks the models read (8).
int NumEncoderLayers { get; set; } The NAR text encoder’s depth.
int PenaltyWindow { get; set; } Tokens the penalties look back over (80, about 5 s).
int Q0OverrideSteps { get; set; } The final diffusion steps at which the NAR may revise the first codebook (20).
int SampleRate { get; set; } The waveform rate, paired with Codec.SampleRate.
int SamplingSeed { get; set; } The seed of sampling and diffusion.
int SlidingWindow { get; set; } The AR decoder’s attention window (3000).
int TimeDim { get; set; } The timestep embedding’s width (1024).
int TopK { get; set; } The AR top-k (200).
int TrainingSeed { get; set; } The training draws’ seed.
int VocabSize { get; set; } Inherited vocabulary metadata; TextTokenizerModel defines the native vocabulary.
int VocosBandwidthId { get; set; } The Vocos bandwidth class decoding uses (1, the 3 kbps class: the release’s smoothing).
int WarmupSteps { get; set; } Warmup steps of both rates (10,000), linear from zero; the rate then falls linearly to its end (the schedule’s shape is not recorded).
string SpeechTokenizerModel { get; set; } The speech tokenizer’s minbpe model (speechtok.model); null for codes with no merges and the end-of-speech special.
string TextTokenizerModel { get; set; } The text tokenizer’s minbpe model (the AR checkpoint’s texttok.model); null for a byte-level tokenizer with no merges and the two text specials.

Methods

Method Summary
EnCodecOptions CreateReleaseCodec() EnCodec’s 24 kHz model at 6 kbps (eight codebooks), as the release encodes with it.
MARS5TTSOptions FromCheckpoint(string arCheckpointPath) The release’s configuration with the tokenizers stored in an AR checkpoint’s metadata (mars5_ar.safetensors).