TrainingMemoryManager<T>
Models & Types · AiDotNet.Training.Memory
Manages memory optimization during neural network training including gradient checkpointing, activation pooling, and model sharding.
Implements: IDisposable
For Beginners
This manager helps you train larger neural networks by:
- Gradient Checkpointing: Saves memory by recomputing activations during backward pass
- Activation Pooling: Reuses tensor memory to reduce garbage collection
- Model Sharding: Distributes layers across multiple GPUs
Example usage:
Constructors
| Constructor | Summary |
|---|---|
TrainingMemoryManager(TrainingMemoryConfig config, IEnumerable<ILayer<T>> layers) |
Initializes a new instance of the TrainingMemoryManager. |
Properties
| Property | Summary |
|---|---|
ActivationPoolStats PoolStats { get; } |
Gets pool statistics if activation pooling is enabled. |
TrainingMemoryConfig Config { get; } |
Gets the memory configuration. |
bool IsCheckpointingEnabled { get; } |
Gets whether gradient checkpointing is enabled. |
bool IsPoolingEnabled { get; } |
Gets whether activation pooling is enabled. |
bool IsShardingEnabled { get; } |
Gets whether model sharding is enabled. |
Methods
| Method | Summary |
|---|---|
MemorySavingsEstimate EstimateMemorySavings(long modelParameters, int batchSize, int sequenceLength) |
Estimates memory savings from current configuration. |
Tensor<T> BackwardSequence(IReadOnlyList<ILayer<T>> layers, Tensor<T> outputGradient) |
Performs backward pass through multiple layers with recomputation. |
Tensor<T> ForwardSequence(IEnumerable<ILayer<T>> layers, Tensor<T> input) |
Performs forward pass through multiple layers with checkpointing. |
Tensor<T> ForwardWithCheckpoint(ILayer<T> layer, Tensor<T> input, int layerIndex) |
Performs a forward pass with checkpointing for a single layer. |
Tensor<T> RentTensor(int[] shape) |
Rents a tensor from the activation pool. |
Tensor<T> ShardedForward(Tensor<T> input) |
Performs forward pass through sharded model. |
bool ShouldCheckpoint(int layerIndex) |
Determines if a specific layer should be checkpointed. |
long GetPoolMemoryUsage() |
Gets current memory usage from the activation pool. |
void ClearCheckpoints() |
Clears all stored checkpoints to free memory. |
void ComputeCheckpointIndices(ILayeredModel<T> layeredModel) |
Determines which layers should be checkpointed using layer-aware metadata from an ILayeredModel. |
void ComputeCheckpointIndices(int totalLayers, IReadOnlyList<string> layerTypes) |
Determines which layers should be checkpointed based on configuration. |
void Dispose() |
Disposes resources used by the memory manager. |
void ReturnTensor(Tensor<T> tensor) |
Returns a tensor to the activation pool. |