SparseLinearLayer<T>
Layers · AiDotNet.NeuralNetworks.Layers
Represents a fully connected layer with sparse weight matrix for efficient computation.
Inheritance: LayerBase<T> → SparseLinearLayer<T>
Implements: IShapeContract
For Beginners
This layer works like a regular dense layer, but uses sparse matrices to store weights more efficiently.
Benefits of sparse layers:
- Much less memory for large layers with few active connections
- Faster computation (only non-zero weights are used)
- Natural for network pruning and compression
Use cases:
- Graph neural networks (sparse adjacency)
- Recommender systems (sparse user-item matrices)
- Pruned neural networks
- Very large embedding layers
How It Works
A sparse linear layer is similar to a dense layer but uses sparse weight storage. This is efficient when most weights are zero (or can be pruned), reducing both memory usage and computation time.
Constructors
| Constructor | Summary |
|---|---|
SparseLinearLayer(int inputFeatures, int outputFeatures, double sparsity, IActivationFunction<T> activationFunction, IInitializationStrategy<T> initializationStrategy) |
Initializes a new instance of the SparseLinearLayer. |
Properties
| Property | Summary |
|---|---|
bool SupportsTraining { get; } |
Gets whether this layer supports training. |
int InputFeatures { get; } |
Gets the number of input features. |
int OutputFeatures { get; } |
Gets the number of output features. |
Methods
| Method | Summary |
|---|---|
Dictionary<string, string> GetParameterRoles() |
Returns parameter roles for per-role optimizer configuration (e.g., weight decay exemption for biases). |
IReadOnlyList<OutputAxisContract> OutputAxesFor(int inputRank) |
|
IReadOnlyList<Tensor<T>> GetTrainableParameters() |
Returns all trainable parameter tensors marked with [TrainableParameter]. |
void ClearGradients() |
Resets the internal state of the layer. |
void ResetState() |
|
void SetTrainableParameters(IReadOnlyList<Tensor<T>> parameters) |
Replaces trainable parameter tensors (e.g., with ParameterBuffer views). |
void UpdateParameters(T learningRate) |
Updates the layer’s parameters using the calculated gradients. |
void ZeroGrad() |
Clears all gradient fields discovered by convention ({paramName}Gradient). |