EmotiVoiceFrontEnd

Models & Types · AiDotNet.TextToSpeech.FrontEnd

EmotiVoice’s bilingual text front end (netease-youdao/EmotiVoice frontend.py, frontend_cn.py, frontend_en.py): Chinese and English text to the released model’s phoneme tokens.

For Beginners

This turns written Chinese and English into the sound symbols EmotiVoice was trained on, such as n i3 sp0 h ao3 for 你好.

How It Works

As the release runs it (g2p_cn_en): Arabic numerals become Chinese numerals (EmotiVoice’s copy of cn2an), the text splits into Chinese runs (Han characters, Chinese punctuation and spaces) and other runs. Chinese runs are cut by jieba; pypinyin reads each word in TONE3 style with pypinyin-dict’s CC-CEDICT phrases loaded, and each syllable splits into an initial and a final (split_py), syllables joined by sp0, words by sp1, a punctuation mark replacing the boundary before it with sp3. Other runs are English: words from LibriSpeech’s lexicon, or, for words it lacks, from g2p_en, each phone in brackets, words joined by engsp1 and engsp4 at punctuation; cn_eng_sp and eng_cn_sp mark a change of language. The sequence is framed by <sos/eos>.

The release’s quirks are kept, for the model was trained on them: a punctuation mark at a Chinese run’s start removes the frame before it, an English word g2p_en reads with a leading mark (a quote) is dropped, and yue splits as y ve.

Properties

Property Summary
EmotiVoiceFrontEnd Default { get; } The shared instance (the dictionaries load once, on first use).
IReadOnlyList<string> Speakers { get; } The released speaker list (a speaker’s id is its index).
IReadOnlyList<string> Tokens { get; } The released token list (502 tokens; a phoneme’s id is its index).

Methods

Method Summary
IReadOnlyList<string> Phonemes(string text) g2p_cn_en: the phonemes of text, framed by <sos/eos>.
int SpeakerId(string name) The id of a speaker of the released list, by its name (such as “8051”).
int[] TokenIds(string text) The token ids of text.
string NumbersToChinese(string text) tn_chinese: each run of digits and periods starting with a digit, as Chinese numerals.