AttentionNetworkOptions
Options & Configuration · AiDotNet.NeuralNetworks.Options
Configuration options for the AttentionNetwork.
Inheritance: ModelOptions → NeuralNetworkOptions → AttentionNetworkOptions
For Beginners
These three numbers control how much machinery the network builds for you.
HeadCountis how many different “points of view” the attention takes at once.EncoderBlockCountis how many times the network re-reads and refines the sequence.FeedForwardExpansionis how much extra room each block gets to think in.
Larger values give the network more capacity and cost more memory and time.
How It Works
These settings describe the default encoder stack that AttentionNetwork
builds when the architecture supplies no explicit layers. The defaults follow Vaswani et al. (2017),
“Attention Is All You Need”: multi-head self-attention with a position-wise feed-forward network whose
inner width is four times the model width.
Constructors
| Constructor | Summary |
|---|---|
AttentionNetworkOptions() |
Properties
| Property | Summary |
|---|---|
int EncoderBlockCount { get; set; } |
Gets or sets the number of stacked transformer encoder blocks. |
int FeedForwardExpansion { get; set; } |
Gets or sets how much wider the feed-forward inner layer is than the embedding. |
int HeadCount { get; set; } |
Gets or sets the number of attention heads in each encoder block. |