InferenceContext<T>

Models & Types · AiDotNet.Memory

Provides a scoped context for inference operations with automatic tensor pooling and lifecycle management. All tensors rented through this context are tracked and automatically returned to the pool when disposed.

Implements: IDisposable

How It Works

InferenceContext simplifies memory management during neural network inference by:

  • Tracking all rented tensors automatically
  • Returning all tensors to the pool on disposal (even if Release wasn’t called)
  • Providing convenient rent methods for common tensor shapes

Basic usage example:

For ambient context support (avoiding parameter threading), use InferenceScope:

Constructors

Constructor Summary
InferenceContext() Initializes a new instance of the InferenceContext class that creates and owns a 256 MB tensor pool.
InferenceContext(int maxPoolSizeMB) Initializes a new instance of the InferenceContext class that creates and owns a tensor pool of the given size.

Properties

Property Summary
bool IsPoolingEnabled { get; set; } Gets or sets whether pooling is enabled.
int RentedTensorCount { get; } Gets the number of tensors currently rented from this context.

Methods

Method Summary
Tensor<T> Rent(int[] shape) Rents a tensor with the specified shape from the pool.
Tensor<T> Rent1D(int length) Rents a 1D tensor (vector) with the specified length.
Tensor<T> Rent2D(int rows, int cols) Rents a 2D tensor (matrix) with the specified dimensions.
Tensor<T> Rent3D(int dim0, int dim1, int dim2) Rents a 3D tensor with the specified dimensions.
Tensor<T> Rent4D(int batch, int channels, int height, int width) Rents a 4D tensor with the specified dimensions (typically for image data).
Tensor<T> RentLike(Tensor<T> template) Rents a tensor with the same shape as the template tensor.
void Dispose() Disposes the context and returns all rented tensors to the pool.
void Release(Tensor<T> tensor) Releases a tensor back to the pool before the context is disposed.