ConstantScaleLayer<T>
Layers · AiDotNet.NeuralNetworks.Layers
Multiplies its input by a fixed (non-trainable) scalar. Useful for paper-canonical embedding rescaling — Vaswani 2017 §3.4 (preserved by T5 / LLaMA / Gemma / Qwen2 / ChatGLM3) multiplies token embeddings by √d_model before feeding them into the transformer stack.
Inheritance: LayerBase<T> → ConstantScaleLayer<T>
Implements: IShapeContract
Constructors
| Constructor | Summary |
|---|---|
ConstantScaleLayer(double scale) |
Properties
| Property | Summary |
|---|---|
bool SupportsTraining { get; } |
Methods
| Method | Summary |
|---|---|
IReadOnlyList<OutputAxisContract> OutputAxesFor(int inputRank) |
Derived from this type’s [TensorLayout] declarations. |
IReadOnlyList<ValueTuple<string, Tensor<T>>> GetRegisteredBuffers() |
|
void ClearGradients() |
|
void ResetState() |