PReLULayer<T>
Layers · AiDotNet.NeuralNetworks.Layers
Implements a Parametric ReLU (PReLU) layer with learnable negative-slope coefficients.
Inheritance: LayerBase<T> → PReLULayer<T>
Implements: IShapeContract
For Beginners
PReLU is like Leaky ReLU except the “leakiness” is learned from data instead of being set by you. For positive inputs the output is unchanged; for negative inputs the output is the input scaled by α. The network adjusts α during training to control how much signal to let through on the negative side. Per-channel α lets different channels learn different leaks, which often works better than a single shared α for convolutional networks.
How It Works
PReLU is a learnable variant of Leaky ReLU where the negative-slope coefficient α is
a trainable parameter rather than a fixed hyperparameter. The function is:
f(x) = max(0, x) + α * min(0, x) = ReLU(x) - α * ReLU(-x)
Two parameterization modes are supported, matching PyTorch’s nn.PReLU:
Introduced in He et al., “Delving Deep into Rectifiers” (2015), which also introduced the Kaiming initialization. The paper’s recommended initial α is 0.25, which is this layer’s default.
Constructors
| Constructor | Summary |
|---|---|
PReLULayer(int numParameters, int channelAxis, double initialAlpha) |
Initializes a new PReLULayer. |
Properties
| Property | Summary |
|---|---|
bool SupportsTraining { get; } |
Gets a value indicating that this layer has trainable parameters (α). |
int ChannelAxis { get; } |
Gets the tensor axis used for per-channel broadcasting. |
int NumParameters { get; } |
Gets the number of independently learned negative slopes. |
Methods
| Method | Summary |
|---|---|
Dictionary<string, string> GetParameterRoles() |
Returns parameter roles for per-role optimizer configuration (e.g., weight decay exemption for biases). |
IReadOnlyList<OutputAxisContract> OutputAxesFor(int inputRank) |
Derived from this type’s [TensorLayout] declarations. |
IReadOnlyList<Tensor<T>> GetTrainableParameters() |
Returns all trainable parameter tensors marked with [TrainableParameter]. |
Tensor<T> GetAlphaTensor() |
Gets the current α tensor. |
void ResetState() |
|
void SetTrainableParameters(IReadOnlyList<Tensor<T>> parameters) |
Replaces trainable parameter tensors (e.g., with ParameterBuffer views). |
void ZeroGrad() |
Clears all gradient fields discovered by convention ({paramName}Gradient). |