QuantizationStrategy

Enums · AiDotNet.Enums

Specifies the quantization strategy (algorithm) to use for model compression. Different strategies offer varying trade-offs between accuracy, speed, and compression ratio.

For Beginners

Think of quantization strategies like different compression algorithms for photos - some preserve more detail, some compress more aggressively. Each strategy uses different math to decide how to convert big numbers (32-bit) to small numbers (8-bit or 4-bit).

How It Works

Strategy Comparison:

Research References:

Fields

Field Summary
QuantizationStrategy AWQ AWQ (Activation-aware Weight Quantization) - protects important weights based on activation magnitudes.
QuantizationStrategy Dynamic Dynamic quantization - computes scale/zero-point at runtime based on actual values.
QuantizationStrategy GPTQ GPTQ (Generative Pre-trained Transformer Quantization) - uses second-order Hessian information to minimize quantization error.
QuantizationStrategy MinMax Simple MinMax quantization - uses minimum and maximum values to determine scale.
QuantizationStrategy QuIPSharp QuIP# (Quantization with Incoherence Processing) - extreme 2-bit quantization using Hadamard transforms and lattice codebooks.
QuantizationStrategy SmoothQuant SmoothQuant - transfers quantization difficulty from activations to weights using mathematical smoothing.
QuantizationStrategy SpinQuant SpinQuant - uses learned rotation matrices to reduce outliers before quantization.
int value__