ProgressiveNeuralNetworks<T, TInput, TOutput>

Models & Types · AiDotNet.ContinualLearning.Strategies

Progressive Neural Networks (PNN) strategy for continual learning.

Inheritance: ContinualLearningStrategyBase<T, TInput, TOutput> → ProgressiveNeuralNetworks<T, TInput, TOutput>

For Beginners

PNN prevents catastrophic forgetting by creating a new “column” (copy of the network) for each new task. Previous columns are frozen, so old knowledge is never overwritten. New columns can use lateral connections to leverage features from previous columns.

How It Works

How it works:

The Math:

For layer l in column k:

h_k^l = f(W_k^l * h_k^(l-1) + Σ_{j<k} U_{k,j}^l * h_j^(l-1))

Where U_{k,j}^l are the lateral connection weights from column j to column k

Comparison to Other Methods:

PNN guarantees no forgetting but has linear memory growth with tasks. Best for scenarios with a small number of tasks where zero forgetting is critical.

Constructors

Constructor Summary
ProgressiveNeuralNetworks(ILossFunction<T> lossFunction) Initializes a new PNN strategy with default options.
ProgressiveNeuralNetworks(ILossFunction<T> lossFunction, PNNOptions<T> options) Initializes a new PNN strategy with custom options.

Properties

Property Summary
T LateralScaling { get; } Gets the lateral connection scaling factor.
bool ModifiesArchitecture { get; }
bool RequiresMemoryBuffer { get; }
int ColumnCount { get; } Gets the number of columns (one per task).
long MemoryUsageBytes { get; }
string Name { get; }

Methods

Method Summary
T ComputeRegularizationLoss(IFullModel<T, TInput, TOutput> model)
ValueTuple<int, bool, int, int> GetColumnInfo(int columnIndex) Gets information about a specific column.
Vector<T> AdjustGradients(IFullModel<T, TInput, TOutput> model, Vector<T> gradients)
Vector<T> ComputeLateralContribution(List<Vector<T>> previousOutputs, int columnIndex) Computes the forward pass through lateral connections.
int GetTotalParameterCount() Gets the total parameter count across all columns.
void FinalizeTask(IFullModel<T, TInput, TOutput> model)
void PrepareForTask(IFullModel<T, TInput, TOutput> model, IDataset<T, TInput, TOutput> taskData)
void Reset()