NaturalSpeechSupPhonemes

Models & Types · AiDotNet.TextToSpeech.CodecBased

Sup-phonemes (Mixed-Phoneme BERT, Zhang et al. 2022, as NaturalSpeech’s phoneme pre-training uses them): byte-pair merges of adjacent phonemes inside words, learned from a phoneme corpus; each phoneme is mapped to the sup-phoneme containing it.

Properties

Property Summary
IReadOnlyList<ValueTuple<int, int, int>> Merges { get; } The merges in order (each pair of ids and its new id).
int VocabSize { get; } The sup-phoneme vocabulary: the phonemes, then one id per merge.

Methods

Method Summary
NaturalSpeechSupPhonemes FromMerges(int phonemes, IEnumerable<ValueTuple<int, int>> merges) Loads merges (each pair and its id, ids consecutive from the phoneme count).
NaturalSpeechSupPhonemes Learn(IEnumerable<int[]> words, int phonemes, int vocabSize) Learns BPE merges until the vocabulary reaches vocabSize (the paper: 30,088) from words given as phoneme-id sequences.
int[] Assign(int[] word) The sup-phoneme id of each phoneme of a word, applying the merges in order.