BlockwiseAnchorLocalErrorCreditRule<T>
Models & Types · AiDotNet.NeuralNetworks.CreditAssignment
Blockwise anchor local error: decoupled block-local sequence credit assignment that partitions a deep
network into contiguous blocks of at most AnchorStride trainable pre-readout layers, places a shared
(full or candidate-sampled) tied-readout next-token loss at the end of every block, and detaches activations
(StopGradient) strictly at block boundaries so every layer inside a block receives multi-layer local credit
while zero gradient crosses between blocks.
Implements: ILocalObjectiveCreditRule<T>
How It Works
Pure layer-by-layer detachment (LayerStride > 1 on TiedReadoutLocalErrorCreditRule) gives
unselected intermediate layers zero gradient, while detaching every single layer (LayerStride = 1) prevents
stacked linear-attention / GatedDeltaNet layers from cooperating with a periodic global attention / MLA anchor.
Partitioning at anchor boundaries (for example AnchorStride = 4 matching a 3:1 GDN:MLA hybrid block) gives
100% of layers constructive block-local credit with zero end-to-end backpropagation.
When StreamBlocksSequentially is true (the default), LocalObjectiveGradientComputer
executes one isolated GradientTape and TensorScope per decoupled block sequentially, bounding peak
activation memory to O(AnchorStride) layers with 0% forward recomputation overhead while producing the exact
same block-decoupled gradients as single-tape execution.
Constructors
| Constructor | Summary |
|---|---|
BlockwiseAnchorLocalErrorCreditRule(int readoutLayerCount, int anchorStride, int maxCandidates, double localWeight, bool detachReadoutOnHiddenBlocks, bool useDepthProgressiveWeight, bool streamBlocksSequentially, int? seed) |
Creates a blockwise anchor local-error credit rule. |
Properties
| Property | Summary |
|---|---|
IReadOnlyList<Tensor<T>> AuxiliaryParameters { get; } |
|
bool DetachReadoutOnHiddenBlocks { get; } |
Gets whether intermediate block anchors detach the shared readout’s own parameters. |
bool IsExactBackprop { get; } |
|
bool StreamBlocksSequentially { get; } |
Gets whether decoupled blocks execute sequentially with O(AnchorStride) activation memory. |
bool UseDepthProgressiveWeight { get; } |
Gets whether intermediate block anchor weights scale quadratically with normalized depth. |
int AnchorStride { get; } |
Gets the number of trainable hidden layers per decoupled anchor block. |
int MaxCandidates { get; } |
Gets the candidate vocabulary subset size U (MaxValue for full vocabulary). |
int ReadoutLayerCount { get; } |
|
string Name { get; } |
Methods
| Method | Summary |
|---|---|
Tensor<T> BuildLocalObjective(LocalObjectiveContext<T> context) |
|
Tensor<T> BuildOutputObjective(LocalObjectiveStep<T> step, Tensor<T> prediction, Tensor<T> defaultLoss) |
|
bool IsIntermediateAnchor(int hiddenIndex, int hiddenLayerCount) |
Returns true when hidden trainable layer hiddenIndex (0-based out of hiddenLayerCount) is the anchor terminating an intermediate decoupled block. |
void ApplyAuxiliaryGradients(IReadOnlyDictionary<Tensor<T>, Tensor<T>> gradients) |
|
void ComputeTeachingSignals(ICreditAssignmentContext<T> context) |
|
void Initialize(ICreditAssignmentContext<T> context) |