MagnitudePruningStrategy<T>

Models & Types · AiDotNet.Pruning

Prunes weights with smallest absolute values.

Implements: IPruningStrategy<T>

For Beginners

This strategy removes the weakest connections in your neural network.

Think of it like trimming weak branches from a tree:

  • Thick, strong branches (large weight values) carry lots of nutrients and stay
  • Thin, weak branches (small weight values) don’t contribute much and get trimmed

In mathematical terms:

  • Each weight gets an importance score equal to its absolute value |w|
  • Weights with the smallest scores are pruned (set to zero)
  • This is simple but surprisingly effective!

For example:

  • A weight of 0.001 has low importance and might be pruned
  • A weight of 0.9 has high importance and will likely be kept
  • A weight of -0.8 has high importance too (|-0.8| = 0.8)

This technique can often remove 50-90% of weights with minimal accuracy loss!

How It Works

Magnitude-based pruning is one of the simplest and most effective pruning strategies. It removes weights with the smallest absolute values, based on the intuition that weights with small magnitudes contribute less to the network’s output.

Constructors

Constructor Summary
MagnitudePruningStrategy() Initializes a new instance of MagnitudePruningStrategy.

Properties

Property Summary
IReadOnlyList<SparsityPattern> SupportedPatterns { get; } Gets supported sparsity patterns.
bool IsStructured { get; } Gets whether this is structured pruning (false for magnitude-based).
bool RequiresGradients { get; } Gets whether this strategy requires gradients (false for magnitude-based).
string Name { get; } Gets the name of this pruning strategy.

Methods

Method Summary
IPruningMask<T> Create2to4Mask(Tensor<T> importanceScores) Creates a 2:4 structured sparsity mask (NVIDIA Ampere compatible).
IPruningMask<T> CreateMask(Matrix<T> importanceScores, double targetSparsity) Creates a pruning mask for tensors by selecting the smallest magnitude weights to prune.
IPruningMask<T> CreateMask(Tensor<T> importanceScores, double targetSparsity) Creates a pruning mask for tensors by selecting the smallest magnitude weights to prune.
IPruningMask<T> CreateMask(Vector<T> importanceScores, double targetSparsity) Creates a pruning mask for tensors by selecting the smallest magnitude weights to prune.
IPruningMask<T> CreateNtoMMask(Tensor<T> importanceScores, int n, int m) Creates an N:M structured sparsity mask.
Matrix<T> ComputeImportanceScores(Matrix<T> weights, Matrix<T> gradients) Computes importance scores as absolute values of weights for tensors.
SparseCompressionResult<T> ToSparseFormat(Tensor<T> weights, SparseFormat format) Converts pruned weights to sparse format for efficient storage.
Tensor<T> ComputeImportanceScores(Tensor<T> weights, Tensor<T> gradients) Computes importance scores as absolute values of weights for tensors.
Vector<T> ComputeImportanceScores(Vector<T> weights, Vector<T> gradients) Computes importance scores as absolute values of weights for tensors.
void ApplyPruning(Matrix<T> weights, IPruningMask<T> mask) Applies the pruning mask to tensor weights in-place.
void ApplyPruning(Tensor<T> weights, IPruningMask<T> mask) Applies the pruning mask to tensor weights in-place.
void ApplyPruning(Vector<T> weights, IPruningMask<T> mask) Applies the pruning mask to tensor weights in-place.