GradientSketchCompressor<T>
Models & Types ยท AiDotNet.FederatedLearning.Compression
Count Sketch-based gradient compression for federated learning.
Inheritance: FederatedLearningComponentBase<T> โ GradientSketchCompressor<T>
For Beginners
A Count Sketch is a compact data structure that approximately stores a large vector by hashing its elements into a smaller table. Think of it as a lossy compression where you can recover the most important elements (top-k) but lose small values to hash collisions.
How It Works
How it works:
Compression ratio: depth * width / gradient_size. With depth=5 and width=gradient_size/100, this achieves ~20x compression.
Constructors
| Constructor | Summary |
|---|---|
GradientSketchCompressor(AdvancedCompressionOptions options) |
Initializes a new instance of GradientSketchCompressor. |
Methods
| Method | Summary |
|---|---|
Tensor<T> Decompress(Matrix<T> sketch, int originalSize, int width) |
Decompresses a Count Sketch back to a gradient tensor using median estimation. |
Tensor<T> DecompressTopK(Matrix<T> sketch, int originalSize, int width, int topK) |
Decompresses only the top-k largest elements from the sketch. |
ValueTuple<Matrix<T>, int, int> Compress(Tensor<T> gradient) |
Compresses a gradient tensor into a Count Sketch. |
double GetCompressionRatio(int originalSize) |
Gets the compression ratio achieved (sketch size / original size). |