DocFormerEncoderLayer<T>
Layers · AiDotNet.NeuralNetworks.Layers
DocFormer’s multi-modal self-attention encoder (reference DocFormerEncoder + MultiModalAttentionLayer).
Inheritance: LayerBase<T> → DocFormerEncoderLayer<T>
Implements: IShapeContract
How It Works
Each layer sees text features t, visual features v, and the two streams’ spatial features
ts, vs, all [L, hidden]. The stream scores are
q k / sqrt(dh) + q . a_lr + k_r . a_lr + q_s k_s / sqrt(dh), where a is the stream’s
clipped relative-position table and the spatial projections q_s, k_s are shared by the two
streams. The layer output is out(softmax(S_t) V_t + softmax(S_v) V_v).
A pre-LN residual wraps each layer. It adds the skip t + v + ts + vs and then the GELU FFN, and only
t is carried to the next layer: v, ts and vs are re-injected unchanged from
layer 0, which is the paper’s multi-modal feature re-use.
Two choices follow the paper rather than the unofficial port. The attention is scaled by
sqrt(head_dim), where the port uses sqrt(hidden). The image stream has its own relative
table, where the port reuses the text table.
The single-input forward takes t with the other streams zero.
Constructors
| Constructor | Summary |
|---|---|
DocFormerEncoderLayer(int hidden, int numHeads, int ffnDim, int maxRelative, int numLayers) |
Creates the encoder. |
Properties
| Property | Summary |
|---|---|
bool SupportsTraining { get; } |
Methods
| Method | Summary |
|---|---|
Dictionary<string, string> GetParameterRoles() |
Returns parameter roles for per-role optimizer configuration (e.g., weight decay exemption for biases). |
IReadOnlyList<ILayer<T>> GetSubLayers() |
|
IReadOnlyList<OutputAxisContract> OutputAxesFor(int inputRank) |
|
IReadOnlyList<Tensor<T>> GetTrainableParameters() |
Returns all trainable parameter tensors marked with [TrainableParameter]. |
void ResetState() |
|
void SetTrainableParameters(IReadOnlyList<Tensor<T>> parameters) |
Replaces trainable parameter tensors (e.g., with ParameterBuffer views). |
void ZeroGrad() |
Clears all gradient fields discovered by convention ({paramName}Gradient). |