CachingCompressor

Models & Types · AiDotNet.PromptEngineering.Compression

Wrapper compressor that caches compression results for frequently used prompts.

Inheritance: PromptCompressorBase → CachingCompressor

For Beginners

Remembers compressions so the same prompt doesn’t need to be compressed twice.

Example:

When to use:

  • Wrapping expensive compressors (LLM-based)
  • When the same prompts are used repeatedly
  • In production systems to reduce API calls

How It Works

This compressor wraps another compressor and caches the results. When the same prompt is compressed multiple times, the cached result is returned instead of re-computing the compression. This is particularly useful for LLM-based compressors where each compression call is expensive.

Constructors

Constructor Summary
CachingCompressor(IPromptCompressor innerCompressor, int maxCacheSize, TimeSpan? cacheExpiry, Func<string, int> tokenCounter) Initializes a new instance of the CachingCompressor class.

Properties

Property Summary
double CacheHitRatio { get; } Gets the cache hit ratio (hits / total requests).
int CacheCount { get; } Gets the number of items currently in the cache.

Methods

Method Summary
Dictionary<string, object> GetCacheStats() Gets statistics about the cache.
Task<string> CompressAsync(string prompt, CompressionOptions options, CancellationToken cancellationToken) Compresses the prompt asynchronously, using cache if available.
int PurgeExpiredEntries() Removes expired entries from the cache.
void ClearCache() Clears all entries from the cache.