WhisperTokenizer

Models & Types · AiDotNet.Audio.Whisper

Tokenizer for Whisper speech recognition model.

For Beginners

A tokenizer converts text to numbers (tokens) and back. Whisper’s tokenizer has special tokens for:

  • Language codes (to specify which language to transcribe)
  • Task tokens (transcribe vs translate)
  • Timestamp tokens (for word-level timing)

How It Works

Whisper uses a special tokenizer with BPE (Byte Pair Encoding) and special tokens for controlling transcription behavior (language, task, timestamps).

Constructors

Constructor Summary
WhisperTokenizer() Initializes a tokenizer that operates in byte-level identity mode (no learned merges).

Properties

Property Summary
IReadOnlyList<string> SupportedLanguages { get; } Gets all supported language codes.
bool StrictBpeMode { get; set; } When true (default false), Encode / Decode throw if the BPE vocab/merges aren’t loaded instead of falling back to the byte-level identity mode.
int EndOfText { get; } Gets the end of text token ID.
int NoSpeechToken { get; } Gets the no speech token ID.
int NoTimestampsToken { get; } Gets the no timestamps token ID.
int StartOfTranscript { get; } Gets the start of transcript token ID.
int TranscribeToken { get; } Gets the transcribe task token ID.
int TranslateToken { get; } Gets the translate task token ID.

Methods

Method Summary
List<long> Encode(string text) Encodes text to GPT-2 / Whisper BPE token IDs.
bool IsSpecialToken(int tokenId) Checks if a token ID is a special token.
bool IsTimestampToken(int tokenId) Checks if a token ID is a timestamp token.
double GetTimeFromToken(int tokenId) Converts a timestamp token ID to time in seconds.
int GetLanguageToken(string languageCode) Gets the token ID for a language code.
int GetTimestampToken(double timeSeconds) Gets the timestamp token ID for a given time in seconds.
string Decode(IEnumerable<long> tokenIds) Decodes a sequence of token IDs back to text.