GradientSketchCompressor<T>

Models & Types ยท AiDotNet.FederatedLearning.Compression

Count Sketch-based gradient compression for federated learning.

Inheritance: FederatedLearningComponentBase<T> โ†’ GradientSketchCompressor<T>

For Beginners

A Count Sketch is a compact data structure that approximately stores a large vector by hashing its elements into a smaller table. Think of it as a lossy compression where you can recover the most important elements (top-k) but lose small values to hash collisions.

How It Works

How it works:

Compression ratio: depth * width / gradient_size. With depth=5 and width=gradient_size/100, this achieves ~20x compression.

Constructors

Constructor Summary
GradientSketchCompressor(AdvancedCompressionOptions options) Initializes a new instance of GradientSketchCompressor.

Methods

Method Summary
Tensor<T> Decompress(Matrix<T> sketch, int originalSize, int width) Decompresses a Count Sketch back to a gradient tensor using median estimation.
Tensor<T> DecompressTopK(Matrix<T> sketch, int originalSize, int width, int topK) Decompresses only the top-k largest elements from the sketch.
ValueTuple<Matrix<T>, int, int> Compress(Tensor<T> gradient) Compresses a gradient tensor into a Count Sketch.
double GetCompressionRatio(int originalSize) Gets the compression ratio achieved (sketch size / original size).