PackNet<T, TInput, TOutput>

Models & Types · AiDotNet.ContinualLearning.Strategies

PackNet strategy for continual learning through network pruning.

Inheritance: ContinualLearningStrategyBase<T, TInput, TOutput> → PackNet<T, TInput, TOutput>

For Beginners

PackNet “packs” multiple tasks into a single network by pruning unimportant parameters after each task and using the freed capacity for new tasks. It’s like having multiple neural networks compressed into one.

How It Works

How it works:

The Math:

After training on task t:

  1. Compute importance: I_i = |θ_i| (magnitude-based) or I_i = |∂L/∂θ_i| * |θ_i|

  2. Prune: mask_t = I > percentile(I, prune_ratio * 100)

  3. Freeze: θ_frozen = θ * mask_t

  4. Free capacity: available = (1 - mask_t)

Comparison to Other Methods:

PackNet has O(1) memory per task but is limited by network capacity. Best for scenarios where network size is constrained and tasks can share features.

Constructors

Constructor Summary
PackNet(ILossFunction<T> lossFunction, PackNetOptions<T> options) Initializes a new PackNet strategy with custom options.
PackNet(ILossFunction<T> lossFunction, double pruningRatio) Initializes a new PackNet strategy with custom options.

Properties

Property Summary
T PruningRatio { get; } Gets the pruning ratio.
bool ModifiesArchitecture { get; }
bool RequiresMemoryBuffer { get; }
double AvailableCapacity { get; } Gets the available parameter capacity (fraction of parameters not yet assigned).
long MemoryUsageBytes { get; }
string Name { get; }

Methods

Method Summary
Dictionary<int, int> GetParameterCountByTask() Gets the count of parameters assigned to each task.
T ComputeRegularizationLoss(IFullModel<T, TInput, TOutput> model)
Vector<T> AdjustGradients(IFullModel<T, TInput, TOutput> model, Vector<T> gradients)
Vector<T> ApplyTaskMask(Vector<T> parameters, int taskId) Applies the task mask to parameters for inference on a specific task.
bool[] GetTaskMask(int taskId) Gets the mask for a specific task (which parameters to use during inference).
int[] GetParameterOwnership() Gets ownership information for all parameters.
void FinalizeTask(IFullModel<T, TInput, TOutput> model)
void PrepareForTask(IFullModel<T, TInput, TOutput> model, IDataset<T, TInput, TOutput> taskData)
void Reset()