HeInitializationStrategy<T>
Models & Types · AiDotNet.Initialization
He/Kaiming initialization strategy for ReLU-family activations.
Inheritance: InitializationStrategyBase<T> → HeInitializationStrategy<T>
For Beginners
Use this when your network uses ReLU or similar activations (which is most modern networks). It’s the PyTorch default for convolutional layers.
How It Works
He initialization accounts for the fact that ReLU zeros out half the values, requiring larger initial weights to maintain variance through the network. This is the recommended strategy for networks using ReLU, Leaky ReLU, GELU, or SiLU.
Constructors
| Constructor | Summary |
|---|---|
HeInitializationStrategy(Random rng, bool useNormal) |
Creates a He initialization strategy with a caller-supplied Random source. |
HeInitializationStrategy(bool useNormal) |
Creates a He initialization strategy. |
Properties
| Property | Summary |
|---|---|
bool IsLazy { get; } |
|
bool LoadFromExternal { get; } |
Methods
| Method | Summary |
|---|---|
IInitializationStrategy<T> WithSeededRandom(Random rng) |
|
void InitializeBiases(Tensor<T> biases) |
|
void InitializeWeights(Tensor<T> weights, int inputSize, int outputSize) |