PowerSGDCompressor<T>
Models & Types · AiDotNet.FederatedLearning.Compression
PowerSGD: low-rank gradient compression using randomized SVD approximation.
Inheritance: FederatedLearningComponentBase<T> → PowerSGDCompressor<T>
For Beginners
Sending a full gradient vector (millions of values) is expensive. PowerSGD compresses it by finding a low-rank approximation — like summarizing a book with just the key plot points. A rank-4 approximation of a 1M-parameter gradient only needs to send ~8K values (500x compression!).
How It Works
How it works:
Warm-start: Reusing P/Q from previous rounds accelerates convergence because the gradient subspace changes slowly between rounds.
Constructors
| Constructor | Summary |
|---|---|
PowerSGDCompressor(AdvancedCompressionOptions options) |
Initializes a new instance of PowerSGDCompressor. |
Methods
| Method | Summary |
|---|---|
Tensor<T> Decompress(Matrix<T> P, Matrix<T> Q, int rows, int cols, int originalSize) |
Decompresses PowerSGD factors back to a gradient tensor. |
ValueTuple<Matrix<T>, Matrix<T>, int, int> Compress(Tensor<T> gradient, int clientId) |
Compresses a gradient tensor using low-rank PowerSGD approximation. |
double GetCompressionRatio(int originalSize) |
Gets the compression ratio achieved (compressed size / original size). |