AgenticChunker
Models & Types · AiDotNet.RetrievalAugmentedGeneration.ChunkingStrategies
Production-ready intelligent chunker that decides where to split text based on semantic boundaries.
Inheritance: ChunkingStrategyBase → AgenticChunker
For Beginners
This is like a smart text splitter that understands content structure.
Think of it like organizing a book:
- Don’t split in the middle of a sentence or paragraph
- Keep related ideas together in the same chunk
- Start new chunks at natural topic boundaries
- Maintain context with overlapping content
How it works:
- Identifies structural elements (paragraphs, sections, lists)
- Calculates semantic coherence scores for potential splits
- Creates chunks at natural boundaries
- Adds overlap for context preservation
Example:
- Input: Long article about climate change
- Output: Chunks at section boundaries, keeping introduction separate from data analysis,
solutions separate from problems, etc.
Unlike simple fixed-size chunking:
- ✓ Respects paragraph boundaries
- ✓ Keeps related sentences together
- ✓ Detects topic changes
- ✓ Preserves document structure
Production features:
- No external API dependencies
- Fast heuristic-based topic detection
- Configurable chunk sizes and overlap
- Handles multiple document formats
- Maintains semantic coherence
How It Works
This chunker analyzes text structure to identify optimal split points based on:
- Paragraph boundaries
- Topic transitions (detected via sentence similarity)
- Natural breaks in content flow (headers, lists, code blocks)
- Semantic coherence within chunks
Constructors
| Constructor | Summary |
|---|---|
AgenticChunker(int maxChunkSize, int overlap, double coherenceThreshold) |
Initializes a new instance of the AgenticChunker class. |