PABCache<T>

Models & Types · AiDotNet.Diffusion.Acceleration

Pyramid Attention Broadcast (PAB) cache for accelerating video diffusion inference.

For Beginners

PAB (Pyramid Attention Broadcast) Cache speeds up video generation by reusing attention computations across nearby diffusion timesteps. Since adjacent steps produce similar attention patterns, caching avoids redundant computation.

How It Works

References:

  • Paper: “Real-Time Video Generation with Pyramid Attention Broadcast” (2024)

PAB exploits the observation that attention outputs in video diffusion models change slowly across denoising timesteps. Instead of recomputing attention at every step, PAB broadcasts (reuses) cached attention outputs for a configurable number of steps:

  • Spatial attention: changes most slowly, can be broadcast for many steps
  • Temporal attention: changes moderately
  • Cross-attention: changes most frequently, broadcast for fewer steps

This pyramid of broadcast intervals achieves significant speedup with minimal quality loss.

Constructors

Constructor Summary
PABCache(int spatialBroadcastInterval, int temporalBroadcastInterval, int crossBroadcastInterval) Initializes a new PAB cache.

Properties

Property Summary
int CrossBroadcastInterval { get; } Gets the cross-attention broadcast interval.
int CurrentStep { get; } Gets the current denoising step.
int SpatialBroadcastInterval { get; } Gets the spatial attention broadcast interval.
int TemporalBroadcastInterval { get; } Gets the temporal attention broadcast interval.

Methods

Method Summary
Tensor<T> GetCachedCross(string layerKey) Gets cached cross-attention output.
Tensor<T> GetCachedSpatial(string layerKey) Gets cached spatial attention output.
Tensor<T> GetCachedTemporal(string layerKey) Gets cached temporal attention output.
bool ShouldRecomputeCross(string layerKey) Checks if cross-attention should be recomputed at the current step.
bool ShouldRecomputeSpatial(string layerKey) Checks if spatial attention should be recomputed at the current step.
bool ShouldRecomputeTemporal(string layerKey) Checks if temporal attention should be recomputed at the current step.
void CacheCross(string layerKey, Tensor<T> output) Caches cross-attention output.
void CacheSpatial(string layerKey, Tensor<T> output) Caches spatial attention output.
void CacheTemporal(string layerKey, Tensor<T> output) Caches temporal attention output.
void Reset() Resets the cache for a new generation.
void Step() Advances to the next denoising step.