PowerSGDCompressor<T>

Models & Types · AiDotNet.FederatedLearning.Compression

PowerSGD: low-rank gradient compression using randomized SVD approximation.

Inheritance: FederatedLearningComponentBase<T> → PowerSGDCompressor<T>

For Beginners

Sending a full gradient vector (millions of values) is expensive. PowerSGD compresses it by finding a low-rank approximation — like summarizing a book with just the key plot points. A rank-4 approximation of a 1M-parameter gradient only needs to send ~8K values (500x compression!).

How It Works

How it works:

Warm-start: Reusing P/Q from previous rounds accelerates convergence because the gradient subspace changes slowly between rounds.

Constructors

Constructor Summary
PowerSGDCompressor(AdvancedCompressionOptions options) Initializes a new instance of PowerSGDCompressor.

Methods

Method Summary
Tensor<T> Decompress(Matrix<T> P, Matrix<T> Q, int rows, int cols, int originalSize) Decompresses PowerSGD factors back to a gradient tensor.
ValueTuple<Matrix<T>, Matrix<T>, int, int> Compress(Tensor<T> gradient, int clientId) Compresses a gradient tensor using low-rank PowerSGD approximation.
double GetCompressionRatio(int originalSize) Gets the compression ratio achieved (compressed size / original size).