TensorRTInferenceEngine<T>

Models & Types · AiDotNet.Deployment.TensorRT

High-performance inference engine for TensorRT models. Supports multi-stream execution and CUDA graph optimization.

Implements: IDisposable

For Beginners

TensorRTInferenceEngine provides AI safety functionality. Default values follow the original paper settings.

Constructors

Constructor Summary
TensorRTInferenceEngine(string enginePath, TensorRTConfiguration config)

Methods

Method Summary
InferenceStatistics GetStatistics() Gets inference statistics for monitoring.
Task WarmUpAsync(int numIterations, int[] inputShape) Warms up the model by running inference on dummy data.
Task<T[]> InferAsync(T[] input) Performs inference on the input data.
Task<T[][]> InferBatchAsync(T[][] inputs) Performs batch inference on multiple inputs concurrently.
void Dispose()
void Initialize() Initializes the inference engine and creates execution contexts.