DiffusionMemoryManager<T>
Models & Types · AiDotNet.Diffusion.Memory
Memory management utilities for diffusion models including gradient checkpointing, activation pooling, and model sharding integration.
For Beginners
Training large models uses a lot of memory because we need to store:
- The model parameters
- Intermediate activations (outputs from each layer during forward pass)
- Gradients for each parameter
This class helps reduce memory usage through several techniques:
Gradient Checkpointing:
- Instead of storing all activations, only store “checkpoints”
- During backward pass, recompute activations between checkpoints
- Trades ~30% more compute time for ~50% less memory
Activation Pooling:
- Reuse tensor memory instead of allocating new tensors
- Reduces GC pressure and memory fragmentation
Model Sharding:
- Split large models across multiple GPUs
- Each GPU only holds part of the model
How It Works
This class provides memory-efficient training utilities specifically designed for large diffusion models (UNet, VAE, etc.) that may not fit in GPU memory during training.
Constructors
| Constructor | Summary |
|---|---|
DiffusionMemoryManager(DiffusionMemoryConfig config, IEnumerable<ILayer<T>> layers) |
Initializes a new instance of the DiffusionMemoryManager class. |
Properties
| Property | Summary |
|---|---|
DiffusionMemoryConfig Config { get; } |
Memory configuration. |
bool CheckpointingEnabled { get; } |
Whether gradient checkpointing is enabled. |
bool PoolingEnabled { get; } |
Whether activation pooling is enabled. |
bool ShardingEnabled { get; } |
Whether model sharding is active. |
Methods
| Method | Summary |
|---|---|
ActivationPoolStats GetPoolStats() |
Gets pooling statistics if available. |
IReadOnlyDictionary<int, long> GetDeviceMemoryUsage() |
Gets memory usage per device. |
MemoryEstimate EstimateMemory(int numLayers, long activationSizeBytes) |
Estimates memory savings from current configuration. |
Tensor<T> RentTensor(int[] shape) |
Rents a tensor from the activation pool. |
Tensor<T> ShardedForward(Tensor<T> input) |
Performs forward pass through sharded model. |
Tensor<T> ShardedForward(Tensor<T> input, Tensor<T> context) |
Performs forward pass through sharded model with context. |
ValueTuple<Tensor<T>, LayerCheckpointState<T>> ForwardWithCheckpointing(IReadOnlyList<ILayer<T>> layers, Tensor<T> input) |
Executes a forward pass through layers with optional checkpointing. |
void ReturnTensor(Tensor<T> tensor) |
Returns a tensor to the activation pool for reuse. |
void ShardedUpdateParameters(T learningRate) |
Updates parameters across all shards. |