QuantizationGranularity
Enums · AiDotNet.Enums
Specifies the granularity level for quantization scaling factors. Finer granularity preserves more accuracy but requires more storage for scale/zero-point values.
For Beginners
When compressing numbers, we need “scaling factors” to convert between big and small numbers. Granularity determines how many different scaling factors we use:
How It Works
Analogy: Think of it like setting brightness on a photo:
Memory Overhead: Finer granularity requires storing more scaling factors:
Research Reference: K-Quant in llama.cpp uses a two-level scheme (PerBlock with super-blocks) achieving excellent quality with minimal overhead.
Fields
| Field | Summary |
|---|---|
QuantizationGranularity PerBlock |
Per-block quantization with super-blocks (K-Quant style from llama.cpp). |
QuantizationGranularity PerChannel |
Per-channel quantization - separate scale and zero-point for each output channel. |
QuantizationGranularity PerGroup |
Per-group quantization - separate scale and zero-point for each group of N consecutive elements. |
QuantizationGranularity PerRow |
Row-wise quantization - separate scale per row of a weight matrix. |
QuantizationGranularity PerTensor |
Per-tensor quantization - single scale and zero-point for the entire tensor. |
int value__ |