LogitSoftcapLayer<T>

Layers · AiDotNet.NeuralNetworks.Layers

Applies logit soft-capping — y = cap · tanh(x / cap) — elementwise, bounding values to (-cap, cap). Used by Gemma-2 on the final logits (and, in the reference, on attention scores).

Inheritance: LayerBase<T> → LogitSoftcapLayer<T>

Implements: IShapeContract

How It Works

The layer has no parameters; it is a fixed nonlinearity placed after the LM head so imported Gemma-2 models reproduce the reference’s bounded output distribution.

Constructors

Constructor Summary
LogitSoftcapLayer(double cap) Creates a logit soft-cap layer.

Properties

Property Summary
bool SupportsTraining { get; }
double Cap { get; } The soft-cap magnitude; outputs lie in (-cap, cap).

Methods

Method Summary
IReadOnlyList<OutputAxisContract> OutputAxesFor(int inputRank) Derived from this type’s [TensorLayout] declarations.
void ClearGradients()
void ResetState()