HybridHuffmanClusteringCompression<T>

Models & Types · AiDotNet.ModelCompression

Implements hybrid compression that combines weight clustering with Huffman encoding.

Inheritance: ModelCompressionBase<T> → HybridHuffmanClusteringCompression<T>

For Beginners

Hybrid compression is like using multiple packing strategies together.

Imagine packing for a trip:

  1. First, you organize similar items together (clustering) - all shirts in one pile, pants in another
  2. Then, you compress each pile differently based on how much you have (Huffman) - frequently used items

get special treatment for quick access

For neural networks, hybrid compression combines two powerful techniques:

Stage 1 - Weight Clustering:

  • Groups similar weights together
  • Replaces millions of unique weights with 256 cluster IDs (much smaller!)
  • Lossy but controlled - you choose the quality/size tradeoff

Stage 2 - Huffman Encoding:

  • Notices that some cluster IDs appear more often than others
  • Gives frequent cluster IDs short codes (like “01”)
  • Gives rare cluster IDs long codes (like “110101”)
  • Lossless - perfectly reversible

Why combine them?

  • Clustering alone: Gets you ~4x compression (32-bit float → 8-bit cluster ID)
  • Huffman alone: Doesn’t help much if all values are unique
  • Together: Clustering creates repeated patterns that Huffman can exploit

Real-world example:

  • Original model: 100 million weights × 32 bits = 3.2 GB
  • After clustering: 100 million cluster IDs × 8 bits = 100 MB (32x compression)
  • After Huffman: 100 million IDs with variable length ≈ 60 MB (53x total compression!)
  • Result: 3.2 GB → 60 MB with <2% accuracy loss

How It Works

Hybrid compression first applies weight clustering to reduce the number of unique values, then uses Huffman encoding on the cluster indices to further compress the data. This two-stage approach can achieve compression ratios of 20-50x or higher while maintaining good accuracy.

Constructors

Constructor Summary
HybridHuffmanClusteringCompression(int numClusters, int maxIterations, double tolerance, int huffmanPrecision, int? randomSeed) Initializes a new instance of the HybridHuffmanClusteringCompression class.

Methods

Method Summary
ValueTuple<Vector<T>, ICompressionMetadata<T>> Compress(Vector<T> weights) Compresses weights using clustering followed by Huffman encoding.
Vector<T> Decompress(Vector<T> compressedWeights, ICompressionMetadata<T> metadata) Decompresses weights by reversing Huffman encoding then clustering.
long GetCompressedSize(Vector<T> compressedWeights, ICompressionMetadata<T> metadata) Gets the total compressed size from both compression stages.