EncodingOptions
Options & Configuration · AiDotNet.Tokenization.Models
Options for encoding text into tokens.
Constructors
| Constructor | Summary |
|---|---|
EncodingOptions() |
Creates default encoding options. |
Properties
| Property | Summary |
|---|---|
bool AddSpecialTokens { get; set; } |
Gets or sets whether to add special tokens (e.g., [CLS], [SEP]). |
bool Padding { get; set; } |
Gets or sets whether to pad sequences to MaxLength. |
bool ReturnAttentionMask { get; set; } |
Gets or sets whether to return attention masks. |
bool ReturnOffsets { get; set; } |
Gets or sets whether to return character offsets. |
bool ReturnPositionIds { get; set; } |
Gets or sets whether to return position IDs. |
bool ReturnTokenTypeIds { get; set; } |
Gets or sets whether to return token type IDs. |
bool Truncation { get; set; } |
Gets or sets whether to truncate sequences that exceed MaxLength. |
int Stride { get; set; } |
Gets or sets the stride for overflow handling (used when truncating). |
int? MaxLength { get; set; } |
Gets or sets the maximum sequence length. |
string PaddingSide { get; set; } |
Gets or sets the padding side (“right” or “left”). |
string TruncationSide { get; set; } |
Gets or sets the truncation side (“right” or “left”). |