ITextToSpeech<T>

Interfaces · AiDotNet.Interfaces

Interface for text-to-speech (TTS) models that synthesize spoken audio from text.

Implements: IFeatureAware, IFullModel<T, Tensor<T>, Tensor<T>>, IGradientComputable<T, Tensor<T>, Tensor<T>>

For Beginners

TTS is like having a computer read text out loud to you.

How TTS works:

  1. Text is analyzed for pronunciation, emphasis, and pacing
  2. The model generates audio features (mel-spectrograms)
  3. A vocoder converts features to waveform audio

Common use cases:

  • Accessibility (screen readers for visually impaired)
  • Voice assistants and chatbots
  • Audiobook and podcast generation
  • Language learning applications

Key features:

  • Voice cloning: Make it sound like a specific person
  • Emotion control: Express happiness, sadness, excitement
  • Speed control: Speak faster or slower

How It Works

Text-to-speech models convert written text into natural-sounding spoken audio. Modern TTS systems use neural networks to produce high-quality, expressive speech that can sound nearly indistinguishable from human speakers.

This interface extends IFullModel for Tensor-based audio processing.

Properties

Property Summary
IReadOnlyList<VoiceInfo<T>> AvailableVoices { get; } Gets the list of available built-in voices.
bool IsOnnxMode { get; } Gets whether this model is running in ONNX inference mode.
bool SupportsEmotionControl { get; } Gets whether this model supports emotional expression control.
bool SupportsStreaming { get; } Gets whether this model supports streaming audio generation.
bool SupportsVoiceCloning { get; } Gets whether this model supports voice cloning from reference audio.
int SampleRate { get; } Gets the sample rate of generated audio.

Methods

Method Summary
IStreamingSynthesisSession<T> StartStreamingSession(string voiceId, double speakingRate) Starts a streaming synthesis session for incremental audio generation.
Task<Tensor<T>> SynthesizeAsync(string text, string voiceId, double speakingRate, double pitch, CancellationToken cancellationToken) Synthesizes speech from text asynchronously.
Tensor<T> ExtractSpeakerEmbedding(Tensor<T> referenceAudio) Extracts speaker embedding from reference audio for voice cloning.
Tensor<T> Synthesize(string text, string voiceId, double speakingRate, double pitch) Synthesizes speech from text.
Tensor<T> SynthesizeWithEmotion(string text, string emotion, double emotionIntensity, string voiceId, double speakingRate) Synthesizes speech with emotional expression.
Tensor<T> SynthesizeWithVoiceCloning(string text, Tensor<T> referenceAudio, double speakingRate, double pitch) Synthesizes speech using a cloned voice from reference audio.