DeploymentRuntime<T>

Models & Types · AiDotNet.Deployment.Runtime

Runtime environment for deployed models with warm-up, versioning, A/B testing, and telemetry.

For Beginners

DeploymentRuntime provides AI safety functionality. Default values follow the original paper settings.

Constructors

Constructor Summary
DeploymentRuntime(RuntimeConfiguration config)

Methods

Method Summary
List<ModelVersionInfo> GetRegisteredModels() Gets all registered model versions.
ModelStatistics GetModelStatistics(string modelName, string version) Gets telemetry statistics for a model.
Task WarmUpModelAsync(string modelName, string version, int numIterations) Warms up a model by running inference on dummy data.
Task<T[]> InferAsync(string modelName, string version, T[] input) Performs inference with the specified model version.
Task<ValueTuple<T[], string>> InferWithABTestAsync(string testName, T[] input) Performs inference with A/B testing (automatically selects version based on traffic split).
void RegisterModel(string modelName, string version, string modelPath, Dictionary<string, object> metadata) Registers a model version with the runtime.
void SetupABTest(string testName, string modelName, string versionA, string versionB, double trafficSplit) Sets up A/B testing between two model versions.