InMemoryEmbeddingCache<T>

Models & Types · AiDotNet.RetrievalAugmentedGeneration.Embeddings

A thread-safe, in-memory IEmbeddingCache backed by a ConcurrentDictionary with optional maximum size and approximate least-recently-used (LRU) eviction.

Implements: IEmbeddingCache<T>

For Beginners

This keeps embeddings in memory so repeated text is not recomputed.

  • It is safe to use from many threads at once.
  • If you set a maximum size, the entries you have not used for the longest time are removed first

once the cache is full (this is called “LRU” eviction).

  • If you do not set a maximum size, it keeps everything until you clear it.

How It Works

When maxSize is zero (or negative) the cache is unbounded. When a positive maxSize is supplied, inserting a new entry that would exceed the limit evicts the least-recently-used entries until the cache is back within bounds. Recency is tracked with a monotonically increasing sequence number that is updated on every read (hit) and write.

Constructors

Constructor Summary
InMemoryEmbeddingCache(int maxSize) Initializes a new instance of the InMemoryEmbeddingCache class.

Properties

Property Summary
int Count { get; }
int MaxSize { get; } Gets the maximum number of entries this cache retains, or 0 if the cache is unbounded.

Methods

Method Summary
bool TryGet(string key, Vector<T> embedding)
void Clear()
void Set(string key, Vector<T> embedding)