TextProcessingHelper
Helpers & Utilities · AiDotNet.Helpers
Provides text processing utilities for splitting and tokenizing text.
Methods
| Method | Summary |
|---|---|
List<string> SplitIntoSentences(string text) |
Splits text into sentences based on common sentence-ending punctuation. |
List<string> Tokenize(string text) |
Tokenizes text by splitting on whitespace and common punctuation marks. |