SparsePruningCompression<T>
Models & Types ยท AiDotNet.ModelCompression
Implements sparse pruning compression by zeroing out small-magnitude weights.
Inheritance: ModelCompressionBase<T> โ SparsePruningCompression<T>
For Beginners
Sparse pruning is like cleaning out your closet.
The idea:
- Many neural network weights are very small (close to zero)
- These tiny weights contribute little to the output
- We can set them to exactly zero without much accuracy loss
- Only store the non-zero weights and their positions
How it works:
- Calculate a threshold (e.g., smallest 90% of weights by magnitude)
- Set all weights below threshold to zero
- Store only non-zero values with their indices
Benefits:
- Can achieve 90%+ sparsity (90% zeros) with minimal accuracy loss
- Sparse storage is very efficient (only store ~10% of weights)
- Works well combined with quantization or clustering
Example:
- Original: [0.001, 0.5, -0.002, 0.8, 0.003, -0.7]
- After 50% pruning: [0, 0.5, 0, 0.8, 0, -0.7]
- Sparse storage: values=[0.5, 0.8, -0.7], indices=[1, 3, 5]
How It Works
Sparse pruning removes weights below a certain threshold, setting them to zero. This creates sparsity in the model which can be exploited for efficient storage using sparse matrix formats (only non-zero values and their indices are stored).
Constructors
| Constructor | Summary |
|---|---|
SparsePruningCompression(double sparsityTarget, double minMagnitudeThreshold, bool useGlobalThreshold) |
Initializes a new instance of the SparsePruningCompression class. |
Methods
| Method | Summary |
|---|---|
ValueTuple<Vector<T>, ICompressionMetadata<T>> Compress(Vector<T> weights) |
Compresses weights by pruning small-magnitude values and storing in sparse format. |
Vector<T> Decompress(Vector<T> compressedWeights, ICompressionMetadata<T> metadata) |
Decompresses sparse weights back to dense format. |
long GetCompressedSize(Vector<T> compressedWeights, ICompressionMetadata<T> metadata) |
Gets the compressed size including sparse values and indices. |