TensorRTInferenceEngine<T>
Models & Types · AiDotNet.Deployment.TensorRT
High-performance inference engine for TensorRT models. Supports multi-stream execution and CUDA graph optimization.
Implements: IDisposable
For Beginners
TensorRTInferenceEngine provides AI safety functionality. Default values follow the original paper settings.
Constructors
| Constructor | Summary |
|---|---|
TensorRTInferenceEngine(string enginePath, TensorRTConfiguration config) |
Methods
| Method | Summary |
|---|---|
InferenceStatistics GetStatistics() |
Gets inference statistics for monitoring. |
Task WarmUpAsync(int numIterations, int[] inputShape) |
Warms up the model by running inference on dummy data. |
Task<T[]> InferAsync(T[] input) |
Performs inference on the input data. |
Task<T[][]> InferBatchAsync(T[][] inputs) |
Performs batch inference on multiple inputs concurrently. |
void Dispose() |
|
void Initialize() |
Initializes the inference engine and creates execution contexts. |