E2TTS<T>
Models & Types · AiDotNet.TextToSpeech.FlowDiffusion
E2 TTS (“Embarrassingly Easy”): fully non-autoregressive text-to-speech by flow-matching speech infilling over characters padded with filler tokens to the speech length, with a flat U-Net Transformer.
Inheritance: NeuralNetworkBase<T> → TtsModelBase<T> → FlowMatchingTtsModelBase<T> → E2TTS<T>
Implements: IAcousticModel<T>
For Beginners
E2 TTS learns to fill in masked parts of a spectrogram from the text alone — no phoneme alignment, no duration model — and to speak it fills in everything after a voice prompt.
How It Works
References: “E2 TTS: Embarrassingly Easy Fully Non-Autoregressive Zero-Shot TTS” (Eskimez et al., SLT
2024); the paper has no public code, so the details it leaves out follow the reproduction in F5-TTS (Chen et al. 2024,
SWivid/F5-TTS UNetT).
The character sequence, padded with the filler token to the frame count, is embedded at the mel width and
concatenated with the noisy mel and the masked mel condition; a projection and a convolutional position embedding feed
a Transformer whose layers i and depth − 1 − i are joined by U-Net skip connections (concatenated and projected), with
the flow-step embedding prepended as a token, RMSNorm pre-normalization, rotary self-attention and GELU feed-forward.
Training and sampling are FlowMatchingTtsModelBase’s.
Constructors
| Constructor | Summary |
|---|---|
E2TTS(NeuralNetworkArchitecture<T> architecture, E2TTSOptions options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer) |
|
E2TTS(NeuralNetworkArchitecture<T> architecture, string modelPath, E2TTSOptions options) |
Properties
| Property | Summary |
|---|---|
int FftSize { get; } |
|
int HopSize { get; } |
|
int MaxTextLength { get; } |
|
int MelChannels { get; } |
Methods
| Method | Summary |
|---|---|
ModelMetadata<T> GetModelMetadata() |
|
Tensor<T> TextToMel(string text) |
Generates a mel spectrogram from text (paired with a Vocos vocoder in the reproduction). |