DeploymentRuntime<T>
Models & Types · AiDotNet.Deployment.Runtime
Runtime environment for deployed models with warm-up, versioning, A/B testing, and telemetry.
For Beginners
DeploymentRuntime provides AI safety functionality. Default values follow the original paper settings.
Constructors
| Constructor | Summary |
|---|---|
DeploymentRuntime(RuntimeConfiguration config) |
Methods
| Method | Summary |
|---|---|
List<ModelVersionInfo> GetRegisteredModels() |
Gets all registered model versions. |
ModelStatistics GetModelStatistics(string modelName, string version) |
Gets telemetry statistics for a model. |
Task WarmUpModelAsync(string modelName, string version, int numIterations) |
Warms up a model by running inference on dummy data. |
Task<T[]> InferAsync(string modelName, string version, T[] input) |
Performs inference with the specified model version. |
Task<ValueTuple<T[], string>> InferWithABTestAsync(string testName, T[] input) |
Performs inference with A/B testing (automatically selects version based on traffic split). |
void RegisterModel(string modelName, string version, string modelPath, Dictionary<string, object> metadata) |
Registers a model version with the runtime. |
void SetupABTest(string testName, string modelName, string versionA, string versionB, double trafficSplit) |
Sets up A/B testing between two model versions. |