HybridHuffmanClusteringCompression<T>
Models & Types · AiDotNet.ModelCompression
Implements hybrid compression that combines weight clustering with Huffman encoding.
Inheritance: ModelCompressionBase<T> → HybridHuffmanClusteringCompression<T>
For Beginners
Hybrid compression is like using multiple packing strategies together.
Imagine packing for a trip:
- First, you organize similar items together (clustering) - all shirts in one pile, pants in another
- Then, you compress each pile differently based on how much you have (Huffman) - frequently used items
get special treatment for quick access
For neural networks, hybrid compression combines two powerful techniques:
Stage 1 - Weight Clustering:
- Groups similar weights together
- Replaces millions of unique weights with 256 cluster IDs (much smaller!)
- Lossy but controlled - you choose the quality/size tradeoff
Stage 2 - Huffman Encoding:
- Notices that some cluster IDs appear more often than others
- Gives frequent cluster IDs short codes (like “01”)
- Gives rare cluster IDs long codes (like “110101”)
- Lossless - perfectly reversible
Why combine them?
- Clustering alone: Gets you ~4x compression (32-bit float → 8-bit cluster ID)
- Huffman alone: Doesn’t help much if all values are unique
- Together: Clustering creates repeated patterns that Huffman can exploit
Real-world example:
- Original model: 100 million weights × 32 bits = 3.2 GB
- After clustering: 100 million cluster IDs × 8 bits = 100 MB (32x compression)
- After Huffman: 100 million IDs with variable length ≈ 60 MB (53x total compression!)
- Result: 3.2 GB → 60 MB with <2% accuracy loss
How It Works
Hybrid compression first applies weight clustering to reduce the number of unique values, then uses Huffman encoding on the cluster indices to further compress the data. This two-stage approach can achieve compression ratios of 20-50x or higher while maintaining good accuracy.
Constructors
| Constructor | Summary |
|---|---|
HybridHuffmanClusteringCompression(int numClusters, int maxIterations, double tolerance, int huffmanPrecision, int? randomSeed) |
Initializes a new instance of the HybridHuffmanClusteringCompression class. |
Methods
| Method | Summary |
|---|---|
ValueTuple<Vector<T>, ICompressionMetadata<T>> Compress(Vector<T> weights) |
Compresses weights using clustering followed by Huffman encoding. |
Vector<T> Decompress(Vector<T> compressedWeights, ICompressionMetadata<T> metadata) |
Decompresses weights by reversing Huffman encoding then clustering. |
long GetCompressedSize(Vector<T> compressedWeights, ICompressionMetadata<T> metadata) |
Gets the total compressed size from both compression stages. |