LearnedTokenTypeEmbeddingLayer<T>

Layers · AiDotNet.NeuralNetworks.Layers

Adds a learned BERT token-type embedding to [S,C] or [B,S,C] states.

Inheritance: LayerBase<T> → LearnedTokenTypeEmbeddingLayer<T>

Implements: IShapeContract

How It Works

The single-report MedCLIP text contract uses segment zero for every token, while retaining BERT’s complete two-row token-type parameter table for checkpoint compatibility.

Constructors

Constructor Summary
LearnedTokenTypeEmbeddingLayer(int tokenTypeCount, int embeddingSize) Creates a BERT-compatible token-type embedding table.

Properties

Property Summary
bool SupportsTraining { get; }
int EmbeddingSize { get; } Gets the embedding width.
int TokenTypeCount { get; } Gets the number of token-type rows retained in the checkpoint.

Methods

Method Summary
Dictionary<string, string> GetParameterRoles() Returns parameter roles for per-role optimizer configuration (e.g., weight decay exemption for biases).
IReadOnlyList<OutputAxisContract> OutputAxesFor(int inputRank) Derived from this type’s [TensorLayout] declarations.
IReadOnlyList<Tensor<T>> GetTrainableParameters() Returns all trainable parameter tensors marked with [TrainableParameter].
void ResetState()
void SetTrainableParameters(IReadOnlyList<Tensor<T>> parameters) Replaces trainable parameter tensors (e.g., with ParameterBuffer views).
void UpdateParameters(T learningRate)
void ZeroGrad() Clears all gradient fields discovered by convention ({paramName}Gradient).