ContextWindowManager

Models & Types · AiDotNet.PromptEngineering

Manages context window limits for LLM prompts, providing token estimation and text truncation utilities.

For Beginners

Large Language Models have a maximum number of tokens they can process at once (the “context window”). This class helps you:

Example:

// Create a manager with 4096 token limit
var manager = new ContextWindowManager(4096);

// Check if your prompt fits
var prompt = "Your long prompt here...";
if (!manager.FitsInWindow(prompt))
{
// Truncate to fit
prompt = manager.TruncateToFit(prompt);
}

// Or split long text into chunks
var longDocument = string.Join(" ", Enumerable.Repeat("some long document text.", 500));
var chunks = manager.SplitIntoChunks(longDocument);

A token is roughly 4 characters or 0.75 words in English, but varies by language and model.

How It Works

This class helps manage the token limits of large language models by providing utilities to estimate token counts, check if text fits within the context window, and truncate or split text that exceeds the limit.

Constructors

Constructor Summary
ContextWindowManager(int maxTokens) Initializes a new instance of the ContextWindowManager with the specified maximum tokens.
ContextWindowManager(int maxTokens, Func<string, int> tokenEstimator) Initializes a new instance of the ContextWindowManager with a custom token estimator.

Properties

Property Summary
int MaxTokens { get; } Gets the maximum number of tokens allowed in the context window.

Methods

Method Summary
List<string> SplitIntoChunks(string text, int overlapTokens) Splits the text into chunks that each fit within the context window.
bool FitsInWindow(string text, int reservedTokens) Checks if the given text fits within the context window.
int EstimateTokens(string text) Estimates the number of tokens in the given text.
int RemainingTokens(string text, int reservedTokens) Calculates the remaining tokens available after accounting for the given text.
string TruncateToFit(string text, int reservedTokens) Truncates the text to fit within the context window.