IQuantizer<T, TInput, TOutput>

Interfaces · AiDotNet.Deployment.Optimization.Quantization

Interface for model quantization strategies. Properly integrates with AiDotNet’s IFullModel architecture.

Known implementations: AWQQuantizer<T, TInput, TOutput>, FP8Quantizer<T, TInput, TOutput>, Float16Quantizer<T, TInput, TOutput>, GPTQQuantizer<T, TInput, TOutput>, Int4Quantizer<T, TInput, TOutput>, Int8Quantizer<T, TInput, TOutput>, MXFP4Quantizer<T, TInput, TOutput>, NF4Quantizer<T, TInput, TOutput>, QuIPSharpQuantizer<T, TInput, TOutput>, SmoothQuantQuantizer<T, TInput, TOutput>, SpinQuantQuantizer<T, TInput, TOutput>

Properties

Property Summary
QuantizationMode Mode { get; } Gets the quantization mode (Int8, Float16, etc.).
int BitWidth { get; } Gets the target bit width for quantization.

Methods

Method Summary
IFullModel<T, TInput, TOutput> Quantize(IFullModel<T, TInput, TOutput> model, QuantizationConfiguration config) Quantizes the model parameters using IFullModel architecture.
double GetScaleFactor(string layerName) Gets the scale factor for a specific layer or parameter.
int GetZeroPoint(string layerName) Gets the zero point for a specific layer or parameter.
void Calibrate(IFullModel<T, TInput, TOutput> model, IEnumerable<TInput> calibrationData) Calibrates the quantizer using calibration data by running forward passes through the model.