OrthogonalInitializationStrategy<T>
Models & Types ยท AiDotNet.Initialization
Orthogonal initialization strategy for RNNs, LSTMs, and deep networks.
Inheritance: InitializationStrategyBase<T> โ OrthogonalInitializationStrategy<T>
For Beginners
Use this for RNNs, LSTMs, and very deep networks. It helps gradients flow smoothly through many layers without vanishing or exploding.
How It Works
Orthogonal initialization creates a random orthogonal matrix, which preserves gradient norms across layers. This prevents vanishing/exploding gradients in deep networks and recurrent architectures.
Constructors
| Constructor | Summary |
|---|---|
OrthogonalInitializationStrategy(double gain) |
Creates an orthogonal initialization strategy. |
Properties
| Property | Summary |
|---|---|
bool IsLazy { get; } |
|
bool LoadFromExternal { get; } |
Methods
| Method | Summary |
|---|---|
void InitializeBiases(Tensor<T> biases) |
|
void InitializeWeights(Tensor<T> weights, int inputSize, int outputSize) |