QuIPSharpQuantizer<T, TInput, TOutput>

Models & Types · AiDotNet.Deployment.Optimization.Quantization.Strategies

QuIP# (Quantization with Incoherence Processing Sharp) quantizer for extreme 2-bit quantization. Uses Hadamard transforms for incoherence and lattice-based codebooks for optimal quantization.

Implements: IQuantizer<T, TInput, TOutput>

For Beginners

QuIP# achieves incredibly aggressive 2-bit quantization (just 4 possible values per weight!) while maintaining reasonable accuracy. It uses mathematical transformations to spread information more evenly before quantization.

How It Works

How It Works:

Key Features:

Reference: Tseng et al., “QuIP#: Even Better LLM Quantization with Hadamard Incoherence and Lattice Codebooks” (2024)

Constructors

Constructor Summary
QuIPSharpQuantizer(QuantizationConfiguration config, int groupSize) Initializes a new instance of the QuIPSharpQuantizer.

Properties

Property Summary
QuantizationMode Mode { get; }
bool IsCalibrated { get; set; } Gets whether the quantizer has been calibrated.
int BitWidth { get; }

Methods

Method Summary
IFullModel<T, TInput, TOutput> Quantize(IFullModel<T, TInput, TOutput> model, QuantizationConfiguration config)
double GetScaleFactor(string layerName)
int GetZeroPoint(string layerName)
void Calibrate(IFullModel<T, TInput, TOutput> model, IEnumerable<TInput> calibrationData)