BidirectionalClozeNetworkLayer<T>
Layers · AiDotNet.NeuralNetworks.Layers
ABINet’s Bidirectional Cloze Network (Fang et al., “Read Like Humans”, CVPR 2021, Sec. 3.3; reference
BCNLanguage): a stack of cross-attention-only transformer decoder layers.
Inheritance: LayerBase<T> → BidirectionalClozeNetworkLayer<T>
Implements: IShapeContract
For Beginners
This is the language model that fixes spelling. Each character position looks at every other character, but never at itself, and guesses what belongs there, like filling in a blank.
How It Works
The input is the projected character-probability sequence. A sinusoidal positional encoding is added to it, and the result is the attention MEMORY (keys and values). The queries are the positional encodings alone, so what a position predicts is built only from its context. The location mask blocks every position from attending to itself (paper Eq. 3), which makes the network a cloze in both directions.
Each sub-layer is the reference TransformerDecoderLayer(self_attn=False), post-norm:
tgt = LN(tgt + MHA(tgt, memory, memory, mask)), then tgt = LN(tgt + FFN(tgt)) with a ReLU
feed-forward. Sub-layer k + 1 queries with sub-layer k’s output, and every sub-layer
attends to the same memory.
Constructors
| Constructor | Summary |
|---|---|
BidirectionalClozeNetworkLayer(int modelDim, int numHeads, int feedForwardDim, int numLayers) |
Creates the network. |
Properties
| Property | Summary |
|---|---|
bool SupportsTraining { get; } |
Methods
| Method | Summary |
|---|---|
IReadOnlyList<ILayer<T>> GetSubLayers() |
|
IReadOnlyList<OutputAxisContract> OutputAxesFor(int inputRank) |
Derived from this type’s [TensorLayout] declarations. |
void ResetState() |