AttentionNetworkOptions

Options & Configuration · AiDotNet.NeuralNetworks.Options

Configuration options for the AttentionNetwork.

Inheritance: ModelOptions → NeuralNetworkOptions → AttentionNetworkOptions

For Beginners

These three numbers control how much machinery the network builds for you.

  • HeadCount is how many different “points of view” the attention takes at once.
  • EncoderBlockCount is how many times the network re-reads and refines the sequence.
  • FeedForwardExpansion is how much extra room each block gets to think in.

Larger values give the network more capacity and cost more memory and time.

How It Works

These settings describe the default encoder stack that AttentionNetwork builds when the architecture supplies no explicit layers. The defaults follow Vaswani et al. (2017), “Attention Is All You Need”: multi-head self-attention with a position-wise feed-forward network whose inner width is four times the model width.

Constructors

Constructor Summary
AttentionNetworkOptions()

Properties

Property Summary
int EncoderBlockCount { get; set; } Gets or sets the number of stacked transformer encoder blocks.
int FeedForwardExpansion { get; set; } Gets or sets how much wider the feed-forward inner layer is than the embedding.
int HeadCount { get; set; } Gets or sets the number of attention heads in each encoder block.