ConstantScaleLayer<T>

Layers · AiDotNet.NeuralNetworks.Layers

Multiplies its input by a fixed (non-trainable) scalar. Useful for paper-canonical embedding rescaling — Vaswani 2017 §3.4 (preserved by T5 / LLaMA / Gemma / Qwen2 / ChatGLM3) multiplies token embeddings by √d_model before feeding them into the transformer stack.

Inheritance: LayerBase<T> → ConstantScaleLayer<T>

Implements: IShapeContract

Constructors

Constructor Summary
ConstantScaleLayer(double scale)

Properties

Property Summary
bool SupportsTraining { get; }

Methods

Method Summary
IReadOnlyList<OutputAxisContract> OutputAxesFor(int inputRank) Derived from this type’s [TensorLayout] declarations.
IReadOnlyList<ValueTuple<string, Tensor<T>>> GetRegisteredBuffers()
void ClearGradients()
void ResetState()