BidirectionalClozeNetworkLayer<T>

Layers · AiDotNet.NeuralNetworks.Layers

ABINet’s Bidirectional Cloze Network (Fang et al., “Read Like Humans”, CVPR 2021, Sec. 3.3; reference BCNLanguage): a stack of cross-attention-only transformer decoder layers.

Inheritance: LayerBase<T> → BidirectionalClozeNetworkLayer<T>

Implements: IShapeContract

For Beginners

This is the language model that fixes spelling. Each character position looks at every other character, but never at itself, and guesses what belongs there, like filling in a blank.

How It Works

The input is the projected character-probability sequence. A sinusoidal positional encoding is added to it, and the result is the attention MEMORY (keys and values). The queries are the positional encodings alone, so what a position predicts is built only from its context. The location mask blocks every position from attending to itself (paper Eq. 3), which makes the network a cloze in both directions.

Each sub-layer is the reference TransformerDecoderLayer(self_attn=False), post-norm: tgt = LN(tgt + MHA(tgt, memory, memory, mask)), then tgt = LN(tgt + FFN(tgt)) with a ReLU feed-forward. Sub-layer k + 1 queries with sub-layer k’s output, and every sub-layer attends to the same memory.

Constructors

Constructor Summary
BidirectionalClozeNetworkLayer(int modelDim, int numHeads, int feedForwardDim, int numLayers) Creates the network.

Properties

Property Summary
bool SupportsTraining { get; }

Methods

Method Summary
IReadOnlyList<ILayer<T>> GetSubLayers()
IReadOnlyList<OutputAxisContract> OutputAxesFor(int inputRank) Derived from this type’s [TensorLayout] declarations.
void ResetState()