IQuantizer<T, TInput, TOutput>
Interfaces · AiDotNet.Deployment.Optimization.Quantization
Interface for model quantization strategies. Properly integrates with AiDotNet’s IFullModel architecture.
Known implementations: AWQQuantizer<T, TInput, TOutput>, FP8Quantizer<T, TInput, TOutput>, Float16Quantizer<T, TInput, TOutput>, GPTQQuantizer<T, TInput, TOutput>, Int4Quantizer<T, TInput, TOutput>, Int8Quantizer<T, TInput, TOutput>, MXFP4Quantizer<T, TInput, TOutput>, NF4Quantizer<T, TInput, TOutput>, QuIPSharpQuantizer<T, TInput, TOutput>, SmoothQuantQuantizer<T, TInput, TOutput>, SpinQuantQuantizer<T, TInput, TOutput>
Properties
| Property | Summary |
|---|---|
QuantizationMode Mode { get; } |
Gets the quantization mode (Int8, Float16, etc.). |
int BitWidth { get; } |
Gets the target bit width for quantization. |
Methods
| Method | Summary |
|---|---|
IFullModel<T, TInput, TOutput> Quantize(IFullModel<T, TInput, TOutput> model, QuantizationConfiguration config) |
Quantizes the model parameters using IFullModel architecture. |
double GetScaleFactor(string layerName) |
Gets the scale factor for a specific layer or parameter. |
int GetZeroPoint(string layerName) |
Gets the zero point for a specific layer or parameter. |
void Calibrate(IFullModel<T, TInput, TOutput> model, IEnumerable<TInput> calibrationData) |
Calibrates the quantizer using calibration data by running forward passes through the model. |