AudioPalmSpeechTokenizer

Enums · AiDotNet.TextToSpeech.MultiModal

How AudioPaLM turns speech into audio tokens (Rubenstein et al. 2023 §3.1).

Fields

Field Summary
AudioPalmSpeechTokenizer W2vBert The paper’s w2v-BERT tokens: a w2v-BERT model’s embeddings, not normalized, clustered by k-means into 1024 tokens at 25 Hz (W2vBert).
AudioPalmSpeechTokenizer W2vBert2 Meta’s public w2v-BERT 2.0 (facebook/w2v-bert-2.0): a hidden layer’s embeddings clustered by k-means (50 Hz), a stand-in for the unreleased USM encoders.
int value__