TransformerSearchSpace<T>
Models & Types ยท AiDotNet.AutoML.SearchSpace
Defines the Transformer-based search space for neural architecture search. Includes attention mechanisms, feed-forward networks, and various architectural choices.
Inheritance: SearchSpaceBase<T> โ TransformerSearchSpace<T>
Constructors
| Constructor | Summary |
|---|---|
TransformerSearchSpace() |
Properties
| Property | Summary |
|---|---|
List<double> DropoutRates { get; set; } |
Dropout rates to search over |
List<int> AttentionHeads { get; set; } |
Number of attention heads to search over |
List<int> FeedForwardMultipliers { get; set; } |
Feed-forward expansion ratios |
List<int> HiddenDimensions { get; set; } |
Hidden dimensions to consider |
bool UsePreNorm { get; set; } |
Whether to use Pre-LayerNorm (true) or Post-LayerNorm (false) |