InMemoryEmbeddingCache<T>
Models & Types · AiDotNet.RetrievalAugmentedGeneration.Embeddings
A thread-safe, in-memory IEmbeddingCache backed by a ConcurrentDictionary
with optional maximum size and approximate least-recently-used (LRU) eviction.
Implements: IEmbeddingCache<T>
For Beginners
This keeps embeddings in memory so repeated text is not recomputed.
- It is safe to use from many threads at once.
- If you set a maximum size, the entries you have not used for the longest time are removed first
once the cache is full (this is called “LRU” eviction).
- If you do not set a maximum size, it keeps everything until you clear it.
How It Works
When maxSize is zero (or negative) the cache is unbounded. When a positive maxSize is
supplied, inserting a new entry that would exceed the limit evicts the least-recently-used entries
until the cache is back within bounds. Recency is tracked with a monotonically increasing sequence
number that is updated on every read (hit) and write.
Constructors
| Constructor | Summary |
|---|---|
InMemoryEmbeddingCache(int maxSize) |
Initializes a new instance of the InMemoryEmbeddingCache class. |
Properties
| Property | Summary |
|---|---|
int Count { get; } |
|
int MaxSize { get; } |
Gets the maximum number of entries this cache retains, or 0 if the cache is unbounded. |
Methods
| Method | Summary |
|---|---|
bool TryGet(string key, Vector<T> embedding) |
|
void Clear() |
|
void Set(string key, Vector<T> embedding) |