QuIPSharpQuantizer<T, TInput, TOutput>
Models & Types · AiDotNet.Deployment.Optimization.Quantization.Strategies
QuIP# (Quantization with Incoherence Processing Sharp) quantizer for extreme 2-bit quantization. Uses Hadamard transforms for incoherence and lattice-based codebooks for optimal quantization.
Implements: IQuantizer<T, TInput, TOutput>
For Beginners
QuIP# achieves incredibly aggressive 2-bit quantization (just 4 possible values per weight!) while maintaining reasonable accuracy. It uses mathematical transformations to spread information more evenly before quantization.
How It Works
How It Works:
Key Features:
Reference: Tseng et al., “QuIP#: Even Better LLM Quantization with Hadamard Incoherence and Lattice Codebooks” (2024)
Constructors
| Constructor | Summary |
|---|---|
QuIPSharpQuantizer(QuantizationConfiguration config, int groupSize) |
Initializes a new instance of the QuIPSharpQuantizer. |
Properties
| Property | Summary |
|---|---|
QuantizationMode Mode { get; } |
|
bool IsCalibrated { get; set; } |
Gets whether the quantizer has been calibrated. |
int BitWidth { get; } |
Methods
| Method | Summary |
|---|---|
IFullModel<T, TInput, TOutput> Quantize(IFullModel<T, TInput, TOutput> model, QuantizationConfiguration config) |
|
double GetScaleFactor(string layerName) |
|
int GetZeroPoint(string layerName) |
|
void Calibrate(IFullModel<T, TInput, TOutput> model, IEnumerable<TInput> calibrationData) |