SynapticIntelligence<T, TInput, TOutput>
Models & Types · AiDotNet.ContinualLearning.Strategies
Synaptic Intelligence (SI) strategy for continual learning.
Inheritance: ContinualLearningStrategyBase<T, TInput, TOutput> → SynapticIntelligence<T, TInput, TOutput>
For Beginners
Synaptic Intelligence is similar to EWC but estimates weight importance online during training rather than computing Fisher Information afterward. It tracks how much each weight contributes to loss reduction using a “path integral”.
How It Works
Key Insight: SI measures how much each parameter contributed to learning, not just how important it is for the current solution. This is done by integrating the gradient signal along the learning trajectory.
How it works:
The Math (Path Integral):
ω_i = Σ_t (-∂L/∂θ_i(t)) × (θ_i(t) - θ_i(t-1))
This approximates the contribution of parameter i to loss reduction.
Advantages over EWC:
Reference: Zenke, F., Poole, B., and Ganguli, S. “Continual Learning Through Synaptic Intelligence” (2017). ICML.
Constructors
| Constructor | Summary |
|---|---|
SynapticIntelligence(ILossFunction<T> lossFunction, SIOptions<T> options) |
Initializes a new SI strategy with custom options. |
SynapticIntelligence(ILossFunction<T> lossFunction, T lambda) |
Initializes a new SI strategy with custom options. |
Properties
| Property | Summary |
|---|---|
IReadOnlyDictionary<int, double> LayerImportance { get; } |
Gets per-layer importance statistics (if tracking enabled). |
T Damping { get; } |
Gets the damping constant. |
T Lambda { get; } |
Gets the regularization strength. |
Vector<T> ConsolidatedImportance { get; } |
Gets a snapshot of the consolidated importance values. |
Vector<T> OptimalParameters { get; } |
Gets a snapshot of the optimal parameters from the last completed task. |
bool IsTrackingTask { get; } |
Gets whether the strategy is currently tracking a task. |
bool ModifiesArchitecture { get; } |
|
bool RequiresMemoryBuffer { get; } |
|
long MemoryUsageBytes { get; } |
|
string Name { get; } |
Methods
| Method | Summary |
|---|---|
T ComputeRegularizationLoss(IFullModel<T, TInput, TOutput> model) |
|
Vector<T> AdjustGradients(IFullModel<T, TInput, TOutput> model, Vector<T> gradients) |
|
void FinalizeTask(IFullModel<T, TInput, TOutput> model) |
|
void NotifyParameterUpdate(Vector<T> currentParameters) |
Notifies SI of a parameter update (call this after optimizer.step()). |
void PrepareForTask(IFullModel<T, TInput, TOutput> model, IDataset<T, TInput, TOutput> taskData) |
|
void Reset() |