Deployment

74 public types in this namespace, organized by kind.

Models & Types (42)

Type Summary
ABTest Represents a single A/B test configuration.
AWQQuantizer<T, TInput, TOutput> AWQ (Activation-aware Weight Quantization) - protects important weights based on activation magnitudes.
ActivationStatistics<T> Holds activation statistics collected during calibration forward passes.
BlockQuantizationState Stores block-wise quantization state for efficient QAT.
CacheStatistics Statistics for the model cache.
CoreMLExporter<T, TInput, TOutput> Exports models to CoreML format for iOS deployment.
DeploymentRuntime<T> Runtime environment for deployed models with warm-up, versioning, A/B testing, and telemetry.
EdgeOptimizer<T, TInput, TOutput> Optimizer for edge device deployment with ARM NEON and other optimizations.
EfficientQATOptimizer<T> EfficientQAT optimizer providing memory-efficient Quantization-Aware Training for large models.
FP8Quantizer<T, TInput, TOutput> FP8 (8-bit Floating Point) quantizer supporting E4M3 and E5M2 formats.
Float16Quantizer<T, TInput, TOutput> FP16 (half-precision) quantizer for model optimization.
GPTQQuantizer<T, TInput, TOutput> GPTQ (Generative Pre-trained Transformer Quantization) - state-of-the-art weight quantization using second-order Hessian information to minimize reconstruction error.
InferenceStatistics Statistics for TensorRT inference engine.
Int4Quantizer<T, TInput, TOutput> INT4 (4-bit) quantizer for model optimization — the simple MinMax counterpart to Int8Quantizer, used for QLoRA-style low-bit base weights.
Int8Quantizer<T, TInput, TOutput> INT8 quantizer for model optimization.
LayerActivationStats<T> Activation statistics for a single layer.
LayerQuantizationParams Per-layer quantization parameters.
MXFP4Quantizer<T, TInput, TOutput> MXFP4 (Microscaling FP4) quantizer - uses shared exponents for efficient 4-bit floating point.
ModelCache<T> Cache for model inference results.
ModelStatistics Statistics for a model.
ModelVersionInfo Public model version information.
NF4Quantizer<T, TInput, TOutput> NF4 (4-bit NormalFloat) quantizer - optimal for normally distributed weights.
NNAPIBackend<T> NNAPI (Neural Networks API) backend for Android deployment.
NNAPIDeviceInfo Information about an NNAPI-capable device.
NNAPIPerformanceInfo Performance information for NNAPI.
OnnxGraph Represents an ONNX computational graph.
OnnxModelExporter<T, TInput, TOutput> Exports AiDotNet models to ONNX format for cross-platform deployment.
OnnxOperation Represents an ONNX operation (node in the computational graph).
OptimizationProfile Represents a TensorRT optimization profile for dynamic shapes.
PartitionedModel<T, TInput, TOutput> Represents a model partitioned for cloud+edge deployment.
QATTrainingHook<T> Quantization-Aware Training (QAT) hook that applies fake quantization during training.
QuIPSharpQuantizer<T, TInput, TOutput> QuIP# (Quantization with Incoherence Processing Sharp) quantizer for extreme 2-bit quantization.
QuantizationState Stores quantization state for a layer during QAT.
SmoothQuantQuantizer<T, TInput, TOutput> SmoothQuant - transfers quantization difficulty from activations to weights using per-channel smoothing.
SpinQuantQuantizer<T, TInput, TOutput> SpinQuant quantizer - uses learned rotation matrices to reduce outliers before quantization.
TFLiteExporter<T, TInput, TOutput> Exports models to TensorFlow Lite format for mobile deployment.
TFLiteTargetSpec TensorFlow Lite target specification with detailed platform requirements.
TelemetryCollector Collects telemetry data for deployed models.
TelemetryEvent Represents a telemetry event.
TensorRTConverter<T, TInput, TOutput> Converts models to TensorRT optimized format for NVIDIA GPU deployment.
TensorRTInferenceEngine<T> High-performance inference engine for TensorRT models.
WeightStreamingReport Telemetry summary for a model’s weight-streaming activity.

Base Classes (1)

Type Summary
ModelExporterBase<T, TInput, TOutput> Abstract base class for model exporters that provides common functionality.

Interfaces (3)

Type Summary
IModelExporter<T, TInput, TOutput> Base interface for model exporters that convert AiDotNet models to various deployment formats.
INNAPIGraphBuilder Translates a model file (TFLite / ONNX / etc.) into an NNAPI operation graph by adding operands and operations to a freshly-created ANeuralNetworksModel handle.
IQuantizer<T, TInput, TOutput> Interface for model quantization strategies.

Enums (6)

Type Summary
CoreMLComputeUnits CoreML compute unit options.
FP8Format Specifies the FP8 format variant to use.
NNAPIDevice NNAPI acceleration devices.
NNAPIExecutionPreference NNAPI execution preferences.
TFLiteTargetType TensorFlow Lite target type for compatibility.
TensorRTPrecision TensorRT precision modes for inference.

Options & Configuration (20)

Type Summary
ABTestingConfig Configuration for A/B testing - comparing multiple model versions by splitting traffic.
AdaptiveInferenceConfig Configuration for adaptive inference.
CacheConfig Configuration for model caching - storing loaded models in memory to avoid repeated loading.
CompressionConfig Configuration for model compression - reducing model size while preserving accuracy.
CoreMLConfiguration Configuration for CoreML model export.
DeploymentConfiguration Aggregates all deployment-related configurations.
EdgeConfiguration Configuration for edge device deployment optimization.
ExportConfig Configuration for exporting models to different formats and platforms.
ExportConfiguration Configuration options for model export operations.
NNAPIConfiguration Configuration for NNAPI backend.
OptimizationProfileConfig Configuration for a single optimization profile (for dynamic shapes).
ProfilingConfig Configuration for performance profiling during model training and inference.
QuantizationConfig Configuration for model quantization - compressing models by using lower precision numbers.
QuantizationConfiguration Configuration for model quantization - comprehensive settings for PTQ and QAT.
RuntimeConfiguration Configuration for the deployment runtime environment.
TFLiteConfiguration Configuration for TensorFlow Lite model export.
TelemetryConfig Configuration for telemetry - tracking and monitoring model inference metrics.
TensorRTConfiguration Configuration for TensorRT model conversion and execution.
VersioningConfig Configuration for model versioning - managing multiple versions of the same model.
WeightStreamingConfig Configuration for weight streaming (paging large model weights to disk when they don’t fit in RAM).

Helpers & Utilities (2)

Type Summary
CalibrationHelper<T, TInput, TOutput> Helper class for calibrating quantizers using real forward passes.
OnnxNode Represents an ONNX node (input or output).