DenseLayer<T>
Layers · AiDotNet.NeuralNetworks.Layers
Represents a fully connected (dense) layer in a neural network.
Inheritance: LayerBase<T> → DenseLayer<T>
Implements: IAuxiliaryLossLayer<T>, IShapeContract
For Beginners
A dense layer is like a voting system where every input gets to vote on every output.
Think of it like this:
- Each input sends information to every output
- Each connection has a different “importance” (weight)
- The layer learns which connections should be strong and which should be weak
For example, in an image recognition task:
- One input might detect a curved edge
- Another might detect a straight line
- The dense layer combines these features to recognize higher-level patterns
Dense layers are the building blocks of many neural networks because they can learn almost any relationship between inputs and outputs, given enough neurons and training data.
How It Works
A dense layer connects every input neuron to every output neuron, with each connection having a learnable weight. This is the most basic and widely used type of neural network layer. Dense layers are capable of learning complex patterns by adjusting these weights during training.
Thread Safety: This layer is not thread-safe. Each layer instance maintains internal state during forward and backward passes. If you need concurrent execution, use separate layer instances per thread or synchronize access to shared instances.
Constructors
| Constructor | Summary |
|---|---|
DenseLayer(int outputSize, IActivationFunction<T> activationFunction, IInitializationStrategy<T> initializationStrategy, BiasMode biasMode) |
Initializes a new instance of the DenseLayer class with the specified input and output sizes and a vector activation function. |
DenseLayer(int outputSize, IVectorActivationFunction<T> vectorActivation, IInitializationStrategy<T> initializationStrategy, BiasMode biasMode) |
Initializes a new instance of the DenseLayer class with the specified input and output sizes and a vector activation function. |
Properties
| Property | Summary |
|---|---|
RegularizationType Regularization { get; set; } |
Gets or sets the type of regularization to apply. |
T AuxiliaryLossWeight { get; set; } |
Gets or sets the weight for the regularization auxiliary loss. |
T L1Strength { get; set; } |
Gets or sets the L1 regularization strength (used when Regularization is L1 or L1L2). |
T L2Strength { get; set; } |
Gets or sets the L2 regularization strength (used when Regularization is L2 or L1L2). |
bool HasUninitializedParameters { get; } |
Gets the total number of trainable parameters in the layer. |
bool IsInitialized { get; } |
|
bool SupportsTraining { get; } |
Gets a value indicating whether this layer supports training through backpropagation. |
bool UseAuxiliaryLoss { get; set; } |
Gets or sets whether auxiliary loss (weight regularization) should be used during training. |
bool UseBias { get; } |
Whether this layer adds its own bias after the matrix product. |
Methods
| Method | Summary |
|---|---|
Dictionary<string, string> GetAuxiliaryLossDiagnostics() |
Gets diagnostic information about the weight regularization auxiliary loss. |
Dictionary<string, string> GetDiagnostics() |
Gets diagnostic information about this component’s state and behavior. |
Dictionary<string, string> GetParameterRoles() |
Returns parameter roles for per-role optimizer configuration (e.g., weight decay exemption for biases). |
IReadOnlyList<OutputAxisContract> OutputAxesFor(int inputRank) |
|
IReadOnlyList<Tensor<T>> GetTrainableParameters() |
Returns all trainable parameter tensors marked with [TrainableParameter]. |
IReadOnlyList<ValueTuple<string, Tensor<T>>> GetRegisteredBuffers() |
|
OnnxLayerOutputs ConvertToOnnx(OnnxGraphBuilder builder, OnnxLayerInputs inputs) |
Emits this Dense layer as an ONNX Gemm node (Y = A·B + C) with the layer’s weights and biases as initializers. |
T ComputeAuxiliaryLoss() |
Computes the auxiliary loss for weight regularization (L1, L2, or both). |
Tensor<T> GetBiases() |
Gets the biases tensor of the layer. |
Tensor<T> GetWeights() |
Gets the weights tensor of the layer. |
Tensor<T> TryComputeFusedLoss(Tensor<T> input, Tensor<T> target, ILossFunction<T> loss) |
This layer as an LM head fused with CrossEntropyWithLogitsLoss: mean CE(input @ W + b, ids) as one differentiable op that keeps no [rows, outputSize] logits on the tape. |
void ClearGradients() |
Clears stored gradients for weights and biases. |
void ResetState() |
Resets the internal state of the layer. |
void SetQuantizedWeightsQ8_0(sbyte[] qs, float[] scales, int inFeatures, int outFeatures) |
Installs a frozen Q8_0 quantized weight for inference. |
void SetTrainableParameters(IReadOnlyList<Tensor<T>> parameters) |
Replaces trainable parameter tensors (e.g., with ParameterBuffer views). |
void UpdateParameters(T learningRate) |
Updates the layer’s parameters (weights and biases) using the calculated gradients. |
void ZeroGrad() |
Clears all gradient fields discovered by convention ({paramName}Gradient). |