LogitSoftcapLayer<T>
Layers · AiDotNet.NeuralNetworks.Layers
Applies logit soft-capping — y = cap · tanh(x / cap) — elementwise, bounding values to
(-cap, cap). Used by Gemma-2 on the final logits (and, in the reference, on attention scores).
Inheritance: LayerBase<T> → LogitSoftcapLayer<T>
Implements: IShapeContract
How It Works
The layer has no parameters; it is a fixed nonlinearity placed after the LM head so imported Gemma-2 models reproduce the reference’s bounded output distribution.
Constructors
| Constructor | Summary |
|---|---|
LogitSoftcapLayer(double cap) |
Creates a logit soft-cap layer. |
Properties
| Property | Summary |
|---|---|
bool SupportsTraining { get; } |
|
double Cap { get; } |
The soft-cap magnitude; outputs lie in (-cap, cap). |
Methods
| Method | Summary |
|---|---|
IReadOnlyList<OutputAxisContract> OutputAxesFor(int inputRank) |
Derived from this type’s [TensorLayout] declarations. |
void ClearGradients() |
|
void ResetState() |