E2TTS<T>

Models & Types · AiDotNet.TextToSpeech.FlowDiffusion

E2 TTS (“Embarrassingly Easy”): fully non-autoregressive text-to-speech by flow-matching speech infilling over characters padded with filler tokens to the speech length, with a flat U-Net Transformer.

Inheritance: NeuralNetworkBase<T> → TtsModelBase<T> → FlowMatchingTtsModelBase<T> → E2TTS<T>

Implements: IAcousticModel<T>

For Beginners

E2 TTS learns to fill in masked parts of a spectrogram from the text alone — no phoneme alignment, no duration model — and to speak it fills in everything after a voice prompt.

How It Works

References: “E2 TTS: Embarrassingly Easy Fully Non-Autoregressive Zero-Shot TTS” (Eskimez et al., SLT 2024); the paper has no public code, so the details it leaves out follow the reproduction in F5-TTS (Chen et al. 2024, SWivid/F5-TTS UNetT).

The character sequence, padded with the filler token to the frame count, is embedded at the mel width and concatenated with the noisy mel and the masked mel condition; a projection and a convolutional position embedding feed a Transformer whose layers i and depth − 1 − i are joined by U-Net skip connections (concatenated and projected), with the flow-step embedding prepended as a token, RMSNorm pre-normalization, rotary self-attention and GELU feed-forward. Training and sampling are FlowMatchingTtsModelBase’s.

Constructors

Constructor Summary
E2TTS(NeuralNetworkArchitecture<T> architecture, E2TTSOptions options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer)
E2TTS(NeuralNetworkArchitecture<T> architecture, string modelPath, E2TTSOptions options)

Properties

Property Summary
int FftSize { get; }
int HopSize { get; }
int MaxTextLength { get; }
int MelChannels { get; }

Methods

Method Summary
ModelMetadata<T> GetModelMetadata()
Tensor<T> TextToMel(string text) Generates a mel spectrogram from text (paired with a Vocos vocoder in the reproduction).