CachingCompressor
Models & Types · AiDotNet.PromptEngineering.Compression
Wrapper compressor that caches compression results for frequently used prompts.
Inheritance: PromptCompressorBase → CachingCompressor
For Beginners
Remembers compressions so the same prompt doesn’t need to be compressed twice.
Example:
When to use:
- Wrapping expensive compressors (LLM-based)
- When the same prompts are used repeatedly
- In production systems to reduce API calls
How It Works
This compressor wraps another compressor and caches the results. When the same prompt is compressed multiple times, the cached result is returned instead of re-computing the compression. This is particularly useful for LLM-based compressors where each compression call is expensive.
Constructors
| Constructor | Summary |
|---|---|
CachingCompressor(IPromptCompressor innerCompressor, int maxCacheSize, TimeSpan? cacheExpiry, Func<string, int> tokenCounter) |
Initializes a new instance of the CachingCompressor class. |
Properties
| Property | Summary |
|---|---|
double CacheHitRatio { get; } |
Gets the cache hit ratio (hits / total requests). |
int CacheCount { get; } |
Gets the number of items currently in the cache. |
Methods
| Method | Summary |
|---|---|
Dictionary<string, object> GetCacheStats() |
Gets statistics about the cache. |
Task<string> CompressAsync(string prompt, CompressionOptions options, CancellationToken cancellationToken) |
Compresses the prompt asynchronously, using cache if available. |
int PurgeExpiredEntries() |
Removes expired entries from the cache. |
void ClearCache() |
Clears all entries from the cache. |