LearnableLogitScaleLayer<T>

Layers · AiDotNet.NeuralNetworks.Layers

Applies exp(clamp(logScale, 0, log(100))) to similarity logits.

Inheritance: LayerBase<T> → LearnableLogitScaleLayer<T>

Implements: IShapeContract

For Beginners

Temperature scaling controls how sharp the image/text similarity distribution is. Learning it lets the model calibrate that sharpness while the clamp prevents unstable magnitudes.

How It Works

Reference: Radford et al., “Learning Transferable Visual Models From Natural Language Supervision”, 2021.

Constructors

Constructor Summary
LearnableLogitScaleLayer(double temperature)

Properties

Property Summary
T Scale { get; }
bool SupportsTraining { get; }

Methods

Method Summary
Dictionary<string, string> GetParameterRoles() Returns parameter roles for per-role optimizer configuration (e.g., weight decay exemption for biases).
IReadOnlyList<OutputAxisContract> OutputAxesFor(int inputRank) Derived from this type’s [TensorLayout] declarations.
IReadOnlyList<Tensor<T>> GetTrainableParameters() Returns all trainable parameter tensors marked with [TrainableParameter].
void ResetState()
void SetTrainableParameters(IReadOnlyList<Tensor<T>> parameters) Replaces trainable parameter tensors (e.g., with ParameterBuffer views).
void UpdateParameters(T learningRate)
void ZeroGrad() Clears all gradient fields discovered by convention ({paramName}Gradient).