TeaCache<T>
Models & Types · AiDotNet.Diffusion.Acceleration
Timestep Embedding Aware Cache (TeaCache) for accelerating video diffusion inference.
For Beginners
TeaCache (Timestep Embedding Aware Cache) accelerates DiT-based video generation by caching and reusing intermediate computations when timestep embeddings are similar. It provides significant speedup with minimal quality loss.
How It Works
References:
- Paper: “TeaCache: Timestep-Aware KV-Cache for Efficient Video Diffusion” (2025)
TeaCache accelerates video diffusion inference by caching and reusing key-value pairs across denoising timesteps. The key insight is that adjacent timesteps have very similar attention patterns, so KV pairs can be reused with minimal quality degradation.
The cache uses a similarity threshold to determine when to recompute:
- If the timestep embedding change is small, reuse cached KV pairs
- If the change exceeds the threshold, recompute attention
- Achieves 2-3x speedup on typical 50-step diffusion sampling
Constructors
| Constructor | Summary |
|---|---|
TeaCache(double reuseThreshold, int maxCacheSize) |
Initializes a new TeaCache. |
Properties
| Property | Summary |
|---|---|
double HitRate { get; } |
Gets the cache hit rate. |
double ReuseThreshold { get; } |
Gets the reuse threshold for timestep embedding similarity. |
int CacheHits { get; } |
Gets the number of cache hits. |
int CacheMisses { get; } |
Gets the number of cache misses. |
int MaxCacheSize { get; } |
Gets the maximum cache size. |
Methods
| Method | Summary |
|---|---|
Tensor<T> Retrieve(string layerKey) |
Retrieves cached KV pairs. |
bool ShouldReuse(string layerKey, double timestepEmbedding) |
Checks if cached KV pairs can be reused for the given layer and timestep. |
void Reset() |
Resets the cache and statistics. |
void Store(string layerKey, Tensor<T> kvPairs, double timestepEmbedding) |
Stores KV pairs in the cache. |