SparseLinearLayer<T>

Layers · AiDotNet.NeuralNetworks.Layers

Represents a fully connected layer with sparse weight matrix for efficient computation.

Inheritance: LayerBase<T> → SparseLinearLayer<T>

Implements: IShapeContract

For Beginners

This layer works like a regular dense layer, but uses sparse matrices to store weights more efficiently.

Benefits of sparse layers:

  • Much less memory for large layers with few active connections
  • Faster computation (only non-zero weights are used)
  • Natural for network pruning and compression

Use cases:

  • Graph neural networks (sparse adjacency)
  • Recommender systems (sparse user-item matrices)
  • Pruned neural networks
  • Very large embedding layers

How It Works

A sparse linear layer is similar to a dense layer but uses sparse weight storage. This is efficient when most weights are zero (or can be pruned), reducing both memory usage and computation time.

Constructors

Constructor Summary
SparseLinearLayer(int inputFeatures, int outputFeatures, double sparsity, IActivationFunction<T> activationFunction, IInitializationStrategy<T> initializationStrategy) Initializes a new instance of the SparseLinearLayer.

Properties

Property Summary
bool SupportsTraining { get; } Gets whether this layer supports training.
int InputFeatures { get; } Gets the number of input features.
int OutputFeatures { get; } Gets the number of output features.

Methods

Method Summary
Dictionary<string, string> GetParameterRoles() Returns parameter roles for per-role optimizer configuration (e.g., weight decay exemption for biases).
IReadOnlyList<OutputAxisContract> OutputAxesFor(int inputRank)
IReadOnlyList<Tensor<T>> GetTrainableParameters() Returns all trainable parameter tensors marked with [TrainableParameter].
void ClearGradients() Resets the internal state of the layer.
void ResetState()
void SetTrainableParameters(IReadOnlyList<Tensor<T>> parameters) Replaces trainable parameter tensors (e.g., with ParameterBuffer views).
void UpdateParameters(T learningRate) Updates the layer’s parameters using the calculated gradients.
void ZeroGrad() Clears all gradient fields discovered by convention ({paramName}Gradient).