EncodingOptions

Options & Configuration · AiDotNet.Tokenization.Models

Options for encoding text into tokens.

Constructors

Constructor Summary
EncodingOptions() Creates default encoding options.

Properties

Property Summary
bool AddSpecialTokens { get; set; } Gets or sets whether to add special tokens (e.g., [CLS], [SEP]).
bool Padding { get; set; } Gets or sets whether to pad sequences to MaxLength.
bool ReturnAttentionMask { get; set; } Gets or sets whether to return attention masks.
bool ReturnOffsets { get; set; } Gets or sets whether to return character offsets.
bool ReturnPositionIds { get; set; } Gets or sets whether to return position IDs.
bool ReturnTokenTypeIds { get; set; } Gets or sets whether to return token type IDs.
bool Truncation { get; set; } Gets or sets whether to truncate sequences that exceed MaxLength.
int Stride { get; set; } Gets or sets the stride for overflow handling (used when truncating).
int? MaxLength { get; set; } Gets or sets the maximum sequence length.
string PaddingSide { get; set; } Gets or sets the padding side (“right” or “left”).
string TruncationSide { get; set; } Gets or sets the truncation side (“right” or “left”).