InferenceContext<T>
Models & Types · AiDotNet.Memory
Provides a scoped context for inference operations with automatic tensor pooling and lifecycle management. All tensors rented through this context are tracked and automatically returned to the pool when disposed.
Implements: IDisposable
How It Works
InferenceContext simplifies memory management during neural network inference by:
- Tracking all rented tensors automatically
- Returning all tensors to the pool on disposal (even if Release wasn’t called)
- Providing convenient rent methods for common tensor shapes
Basic usage example:
For ambient context support (avoiding parameter threading), use InferenceScope:
Constructors
| Constructor | Summary |
|---|---|
InferenceContext() |
Initializes a new instance of the InferenceContext class that creates and owns a 256 MB tensor pool. |
InferenceContext(int maxPoolSizeMB) |
Initializes a new instance of the InferenceContext class that creates and owns a tensor pool of the given size. |
Properties
| Property | Summary |
|---|---|
bool IsPoolingEnabled { get; set; } |
Gets or sets whether pooling is enabled. |
int RentedTensorCount { get; } |
Gets the number of tensors currently rented from this context. |
Methods
| Method | Summary |
|---|---|
Tensor<T> Rent(int[] shape) |
Rents a tensor with the specified shape from the pool. |
Tensor<T> Rent1D(int length) |
Rents a 1D tensor (vector) with the specified length. |
Tensor<T> Rent2D(int rows, int cols) |
Rents a 2D tensor (matrix) with the specified dimensions. |
Tensor<T> Rent3D(int dim0, int dim1, int dim2) |
Rents a 3D tensor with the specified dimensions. |
Tensor<T> Rent4D(int batch, int channels, int height, int width) |
Rents a 4D tensor with the specified dimensions (typically for image data). |
Tensor<T> RentLike(Tensor<T> template) |
Rents a tensor with the same shape as the template tensor. |
void Dispose() |
Disposes the context and returns all rented tensors to the pool. |
void Release(Tensor<T> tensor) |
Releases a tensor back to the pool before the context is disposed. |