LayerBase<T>

Layers · AiDotNet.NeuralNetworks.Layers

Represents the base class for all neural network layers, providing common functionality and interfaces.

Implements: ICloneable<LayerBase<T>>, IDisposable, IParameterChunkSource<T>, IParameterLayoutSource, IParameterSurfaceLifecycle, ITrainableLayer<T>, IUpstreamBiasRedundancy

Known derived types: ABCLayer<T>, AConvBlock<T>, ADownBlock<T>, ALiBiPositionalBiasLayer<T>, AcousticConditionEncoderLayer<T>, ActNormFlowLayer<T>, ActivationLayer<T>, AdaptiveAveragePoolingLayer<T>, AddLayer<T>, AffineCouplingFlowLayer<T>, AnomalyDetectorLayer<T>, AttentionLayer<T>, AttentionalFeatureFusionLayer<T>, AttentiveTransformerLayer<T>, AutoregressiveEncoderDecoderLayer<T>, AveragePoolingLayer<T>, BASEDLayer<T>, BartEncoderDecoderLayer<T>, BasicBlock<T>, BatchEnsembleLayer<T>, BatchNormalizationLayer<T>, BayesianDenseLayer<T>, BertEmbeddingLayer<T>, BiaffineSpanScorerLayer<T>, BiasFreeLinearLayer<T>, BidirectionalClozeNetworkLayer<T>, BidirectionalLayer<T>, BidirectionalRecurrentLayer<T>, BottleneckBlock<T>, BranchformerBlock<T>, C2PSABlock<T>, C2fBlock<T>, C3k2Block<T>, C3kBlock<T>, CSPBlock<T>, CSPBottleneckBlock<T>, CachedGroupedQueryAttention<T>, CachedMultiHeadAttention<T>, CapsuleLayer<T>, CausalTemporalAttention<T>, CbhgLayer<T>, CifAlignmentLayer<T>, CitrinetBlockLayer<T>, ClipVisionTransformerLayer<T>, ClozeAttentionLayer<T>, CohereDecoderBlock<T>, ColumnParallelLinear<T>, ConcatenateLayer<T>, ConditionalLayerNormalizationLayer<T>, ConditionalRandomFieldLayer<T>, ConformerBlockLayer<T>, ConstantScaleLayer<T>, ContextNetBlockLayer<T>, ContinuumMemorySystemLayer<T>, Conv1DLayer<T>, Conv1DTransposeLayer<T>, Conv2DLayer<T>, Conv3DLayer<T>, Conv3DTransposeLayer<T>, ConvBatchNormStackLayer<T> (+312 more)

For Beginners

This is the blueprint that all neural network layers follow.

Think of LayerBase as the common foundation that all layers are built upon:

  • It defines what every layer must be able to do (process data forward and backward)
  • It provides shared tools that all layers can use (like activation functions)
  • It manages the shapes of data flowing in and out of layers
  • It handles saving and loading layer parameters

All specific layer types (like convolutional, dense, etc.) inherit from this class, which ensures they all work together consistently in a neural network.

How It Works

LayerBase is an abstract class that serves as the foundation for all neural network layers. It defines the common structure and functionality that all layers must implement, such as forward and backward propagation, parameter management, and activation functions. This class handles the core mechanics of layers in a neural network, allowing derived classes to focus on their specific implementations.

Properties

Property Summary
IActivationFunction<T> ScalarActivation { get; set; } Gets the element-wise activation function for this layer, if specified.
IInitializationStrategy<T> InitializationStrategy { get; set; } Gets or sets the initialization strategy for this layer.
IReadOnlyList<LayerPort> InputPorts { get; }
IReadOnlyList<LayerPort> OutputPorts { get; }
IVectorActivationFunction<T> VectorActivation { get; set; } Gets the vector activation function for this layer, if specified.
NumericalReproducibility Reproducibility { get; set; } Whether this layer must reproduce its reference computation exactly, or may take the fastest route available.
Vector<T> ScatteredParameterGradients { get; } Populates this layer’s gradient surface from tape-computed gradients.
bool CanExecuteOnGpu { get; } Gets whether this layer can execute its forward pass on GPU.
bool CanTrainOnGpu { get; } Gets whether this layer can execute GPU training (forward, backward, parameter update).
bool DeterministicForward { get; set; } When true, layers that have a faster but numerically-different inference path (e.g.
bool HasUninitializedParameters { get; } true when this layer will have trainable parameters but cannot size them yet.
bool IsInitialized { get; } Gets a value indicating whether this layer has been initialized.
bool IsShapeResolved { get; }
bool MakesUpstreamBiasRedundant { get; }
bool PropagatesInputDomain { get; } True when this layer hands its input to the NEXT layer without altering the values, so the caller’s tensor must satisfy that next layer’s domain rather than this one’s.
bool RequiresMultipleInputs { get; } Performs the forward pass of the layer with multiple input tensors.
bool SupportsGpuTraining { get; } Gets whether this layer has full GPU training support (forward, backward, and parameter updates).
bool SupportsTraining { get; } Gets a value indicating whether this layer supports training.
bool UseAutodiff { get; set; } Gets or sets a value indicating whether this layer uses automatic differentiation for backward passes.
double LearningRateScale { get; set; } Multiplies the optimizer’s learning rate for this layer’s own trainable parameters.
double? MaxLearningRate { get; set; } The largest learning rate the optimizer may apply to this layer’s own trainable parameters, or null for none.
int NamedParameterCount { get; } Gets the total number of named parameters.
int? RandomSeed { get; set; }
long ParameterCount { get; }
string LayerName { get; } Gets the name of this layer for mixed-precision policy lookup.

Methods

Method Summary
BoundInputContract BindInputContract(int[] inputShape, string variant) Binds this layer’s input contract to a concrete primary-input shape.
Dictionary<string, string> GetDiagnostics() Gets diagnostic information about this layer’s state and behavior.
IEnumerable<ActivationFunction> GetActivationTypes() Gets the types of activation functions used by this layer.
IEnumerable<string> GetParameterNames() Gets all parameter names in this layer.
IReadOnlyList<ILayer<T>> GetSubLayers() Returns all child layers registered via RegisterSubLayer.
IReadOnlyList<ParameterSlotDescriptor> GetParameterLayout()
IReadOnlyList<Tensor<T>> GetTrainableParameters() Returns all trainable parameter tensors registered via RegisterTrainableParameter.
IReadOnlyList<ValueTuple<string, Tensor<T>>> GetRegisteredBuffers() Gets all registered buffers (non-trainable persistent tensors) for this layer.
InputContractManifest GetInputContract(int[] inputShape) Returns the canonical generated/runtime input contract for this layer.
LayerBase<T> Clone() Creates a complete, independent copy of this layer.
LayerBase<T> To(DeviceInfo device) Moves this layer’s parameters and buffers (and, recursively, every registered sub-layer) to the given device — the layer-level equivalent of PyTorch’s module.to(device).
LayerCategory GetLayerCategory() Gets the category classification for this layer, used by automated per-layer tools like quantizers, pruners, pipeline partitioners, and LoRA adapters.
LayerInputDomain GetInputDomain(int[] inputShape) What this layer accepts in its input tensor: continuous values, or integer indices in a bounded range.
LayerShape GetOutputLayerShape()
OnnxLayerOutputs ConvertToOnnx(OnnxGraphBuilder builder, OnnxLayerInputs inputs) Emits this layer as one or more ONNX nodes into builder.
Tensor<T> Forward(IReadOnlyDictionary<string, Tensor<T>> inputs) Performs the forward pass of the layer.
Tensor<T> Forward(Tensor<T> input) Performs the forward pass of the layer.
Tensor<T> Forward(Tensor<T>[] inputs) Performs the forward pass of the layer.
Tensor<T> ForwardGpu(IReadOnlyDictionary<string, Tensor<T>> inputs) Performs the forward pass of the layer on GPU: the same computation as Forward.
Tensor<T> ForwardGpu(Tensor<T>[] inputs) Performs the forward pass of the layer on GPU: the same computation as Forward.
Tensor<T> ForwardWithPrecisionCheck(Tensor<T> input) Performs a forward pass with automatic mixed-precision handling.
Tensor<T> GetBiases() Gets the bias tensor for layers that have trainable biases.
Tensor<T> GetWeights() Gets the weight matrix for layers that have trainable weights.
Vector<T> GetParameterGradients() Gets the gradients of all trainable parameters in this layer.
Vector<T> GetParameters()
WeightLoadResult LoadWeights(Dictionary<string, Tensor<T>> weights, Func<string, string> mapping, bool strict) Loads weights from a dictionary of tensors using optional name mapping.
WeightLoadValidation ValidateWeights(IEnumerable<string> weightNames, Func<string, string> mapping) Validates that a set of weight names can be loaded into this layer.
bool SetParameter(string name, Tensor<T> value) Sets a parameter tensor by name.
bool TryGetParameter(string name, Tensor<T> tensor) Tries to get a parameter tensor by name.
int ScatterParameterGradients(IReadOnlyDictionary<Tensor<T>, Tensor<T>> grads)
int[] GetInputShape() Gets the input shape for this layer.
int[] GetOutputShape() Gets the output shape for this layer.
int[] GetParameterShape(string name) Gets the expected shape for a parameter.
int[][] GetInputShapes() Gets all input shapes for this layer.
long EstimateActivationMemory() Estimates the activation memory (in bytes) needed during a forward pass.
long EstimateFlops() Estimates the computational cost (FLOPs) for a single forward pass through this layer.
void ClearGradients() Clears all parameter gradients in this layer.
void Deserialize(BinaryReader reader) Deserializes the layer’s parameters from a binary reader.
void Dispose() Releases all resources used by this layer, including any GPU resources.
void DownloadWeightsFromGpu() Downloads the layer’s weights and biases from GPU memory back to CPU.
void ResetState() Resets the internal state of the layer.
void ResolveFromShape(int[] inputShape) Eagerly resolves a lazy layer’s shape (and allocates its weights) from a known input shape, without running an actual forward pass.
void ResolveShapesOnly(int[] inputShape) Resolves a lazy layer’s InputShape and OutputShape from a concrete input shape WITHOUT allocating or initializing weights.
void Serialize(BinaryWriter writer) Serializes the layer’s parameters to a binary writer.
void SetDeterministicForward(bool deterministic) Sets DeterministicForward on this layer and recursively on every registered sub-layer (mirrors SetTrainingMode’s propagation), so a composite layer can put its embedded layers on the unfused, train/eval-identical forward path with a sin…
void SetParameters(Vector<T> parameters) Sets the trainable parameters of the layer.
void SetTrainableParameters(IReadOnlyList<Tensor<T>> parameters) Replaces this layer’s trainable parameter tensors with the provided tensors.
void SetTrainingMode(bool isTraining) Sets whether the layer is in training mode or inference mode.
void UpdateParameters(T learningRate) Updates the parameters of the layer with the given vector of parameter values.
void UpdateParameters(Vector<T> parameters) Updates the parameters of the layer with the given vector of parameter values.
void UpdateParametersGpu(IGpuOptimizerConfig config) Updates the layer’s parameters on GPU using the specified optimizer configuration.
void UploadWeightsToGpu() Uploads the layer’s weights and biases to GPU memory for GPU-resident training.
void ZeroGrad() Clears all accumulated gradients.
void ZeroGradientsGpu() Resets the GPU gradient accumulators to zero.