FeedForwardLayer<T>
Layers · AiDotNet.NeuralNetworks.Layers
Represents a fully connected (dense) feed-forward layer in a neural network.
Inheritance: LayerBase<T> → FeedForwardLayer<T>
Implements: IShapeContract
For Beginners
A feed-forward layer is like a voting system where every input gets to vote on every output.
Imagine you have 3 inputs and 2 outputs:
- Each input has a different level of influence (weight) on each output
- Each output has its own starting value (bias)
- The layer calculates each output by combining all input influences plus the bias
- Finally, an activation function adds non-linearity (like setting a threshold)
For example:
- Input: [0.2, 0.5, 0.1] (representing features from previous layer)
- Weights: [[0.1, 0.8], [0.4, 0.3], [0.7, 0.2]] (each input’s influence on each output)
- Biases: [0.1, -0.2] (starting values for each output)
- Output before activation: [0.2×0.1 + 0.5×0.4 + 0.1×0.7 + 0.1, 0.2×0.8 + 0.5×0.3 + 0.1×0.2 - 0.2]
= [0.39, 0.33]
- After activation (e.g., ReLU): [0.39, 0.33] (since both are already positive)
Feed-forward layers are the building blocks of many neural networks. Multiple feed-forward layers stacked together form a “deep” neural network that can learn increasingly complex patterns.
How It Works
A feed-forward layer, also known as a fully connected or dense layer, is one of the most common types of neural network layers. It connects every input neuron to every output neuron with learnable weights. Each output neuron also has a learnable bias term. The layer applies a linear transformation followed by an activation function to produce its output.
Constructors
| Constructor | Summary |
|---|---|
FeedForwardLayer(int outputSize, IActivationFunction<T> activationFunction) |
Initializes a new instance of the FeedForwardLayer class with a vector activation function. |
FeedForwardLayer(int outputSize, IVectorActivationFunction<T> activationFunction) |
Initializes a new instance of the FeedForwardLayer class with a vector activation function. |
Properties
| Property | Summary |
|---|---|
bool HasUninitializedParameters { get; } |
Gets the total number of trainable parameters in this layer. |
bool IsInitialized { get; } |
Gets whether this layer’s parameters have been allocated and initialized. |
bool SupportsTraining { get; } |
The computation engine (CPU or GPU) for vectorized operations. |
Methods
| Method | Summary |
|---|---|
Dictionary<string, string> GetParameterRoles() |
Returns parameter roles for per-role optimizer configuration (e.g., weight decay exemption for biases). |
IReadOnlyList<OutputAxisContract> OutputAxesFor(int inputRank) |
|
IReadOnlyList<Tensor<T>> GetTrainableParameters() |
Returns all trainable parameter tensors marked with [TrainableParameter]. |
Tensor<T> GetBiasesTensor() |
Gets the bias tensor for JIT compilation and graph composition. |
Tensor<T> GetWeightsTensor() |
Gets the weight tensor for JIT compilation and graph composition. |
void ClearGradients() |
|
void ResetState() |
|
void SetTrainableParameters(IReadOnlyList<Tensor<T>> parameters) |
Replaces trainable parameter tensors (e.g., with ParameterBuffer views). |
void UpdateParameters(T learningRate) |
Updates the weights and biases using the calculated gradients and the specified learning rate. |
void ZeroGrad() |
Clears all gradient fields discovered by convention ({paramName}Gradient). |