DefaultGradientCache<T>

Models & Types · AiDotNet.Caching

Provides a default implementation of the gradient caching mechanism for symbolic models.

Implements: IGradientCache<T>

For Beginners

A gradient cache is like a memory bank that stores pre-calculated mathematical operations that are frequently used during machine learning model training.

In machine learning, “gradients” are calculations that show how much a model’s error would change if we slightly adjusted a specific parameter. These calculations can be complex and time-consuming, especially if they need to be repeated many times.

By storing these calculations in a cache (a temporary storage area), we can avoid recalculating the same gradients repeatedly, which makes the training process much faster. Think of it like remembering the answer to a difficult math problem so you don’t have to solve it again when you need the same answer later.

This class provides a simple way to store and retrieve these gradient calculations using string keys (like names or identifiers) to look them up quickly.

How It Works

Bounded capacity (memory-leak fix): the cache evicts its oldest entry once the number of stored gradients exceeds Capacity. This matters because the optimizer’s gradient cache key (GradientBasedOptimizerBase.GenerateGradientCacheKey) folds in a per-step parameter-state fingerprint AND the per-batch input/target reference identities. During the batched Optimize training loop every mini-batch step therefore produces a BRAND-NEW key that will never be looked up again (the parameters have already moved on by the next step). Without a bound the dictionary grew by one full parameter-sized gradient Vector<T> every step and was only ever cleared by Reset() (which the training loop never calls) — a linear managed-heap leak that made each epoch’s GC progressively more expensive and per-epoch wall-time climb roughly linearly with the epoch index. Bounding the cache is numerically transparent: a cache hit returns the identical gradient that would otherwise be recomputed, and a miss (because an old entry was evicted) simply recomputes that same gradient deterministically. The legitimate reuse cases the cache exists for — line search / trust-region re-evaluation of the same solution, or a caller invoking CalculateGradient twice in a row with identical arguments — all reuse a key within a handful of consecutive calls, well inside the retained window, so they still hit.

Constructors

Constructor Summary
DefaultGradientCache() Creates a gradient cache with the recommended defaults: DefaultCapacity entries, DefaultEvictionPolicy eviction.
DefaultGradientCache(int capacity) Creates a gradient cache bounded to capacity entries with the recommended DefaultEvictionPolicy eviction.
DefaultGradientCache(int capacity, CacheEvictionPolicy evictionPolicy, bool enabled) Creates a gradient cache bounded to capacity entries evicting per evictionPolicy.

Properties

Property Summary
CacheEvictionPolicy EvictionPolicy { get; } The policy used to choose the eviction victim once the cache is full.
bool Enabled { get; } Whether the cache actually stores entries (false = disabled pass-through).
int Capacity { get; } Maximum number of gradient entries retained before an entry is evicted.

Methods

Method Summary
IGradientModel<T> GetCachedGradient(string key) Retrieves a cached gradient model using the specified key.
void CacheGradient(string key, IGradientModel<T> gradient) Stores a gradient model in the cache with the specified key.
void ClearCache() Removes all cached gradient models from the cache.

Fields

Field Summary
CacheEvictionPolicy DefaultEvictionPolicy Recommended default eviction policy: oldest-inserted first.
int DefaultCapacity Default maximum number of gradient entries retained before the oldest is evicted.