HouseholderSubspaceLayer<T>

Layers · AiDotNet.NeuralNetworks.Layers

Matrix-free orthogonal projection y = x Q (or y = x Qᵀ) parameterized as a product of k Householder reflections Q = H₀ H₁ ⋯ H_{k−1}, where each Hⱼ = I − (2 / (‖uⱼ‖₂² + ε)) uⱼ uⱼᵀ uses only O(k · D) parameters and 4 k · M · D FLOPs without ever materializing a D × D matrix.

Inheritance: LayerBase<T> → HouseholderSubspaceLayer<T>

Implements: IShapeContract

How It Works

Every non-zero reflector vector uⱼ ∈ ℝᴰ defines an exact symmetric orthogonal reflection (Hⱼᵀ = Hⱼ, Hⱼ² = I). Consequently Qᵀ Q = I holds by construction for arbitrary unconstrained gradient updates to uⱼ without SVD, QR retraction, or Cayley matrix inversions, preserving activation and gradient Euclidean norms across arbitrarily deep shared-parameter stacks (e.g. 100+ layers).

Setting Transpose to true applies the reflections in reverse order (H_{k−1} ⋯ H₁ H₀ = Qᵀ = Q⁻¹), providing the exact inverse/transpose rotation.

Constructors

Constructor Summary
HouseholderSubspaceLayer(int dimension, int numReflectors, bool transpose) Creates a matrix-free orthogonal Householder subspace rotation layer of width dimension using numReflectors reflections.

Properties

Property Summary
Tensor<T> Reflectors { get; } Gets the underlying [NumReflectors, Dimension] trainable reflector matrix.
bool SupportsTraining { get; }
bool Transpose { get; } Gets whether the reflection chain is applied in reverse order (Qᵀ = Q⁻¹).
int Dimension { get; } Gets the feature dimension D rotated by this layer.
int NumReflectors { get; } Gets the number of Householder reflections k composed by this layer.

Methods

Method Summary
Dictionary<string, string> GetParameterRoles() Returns parameter roles for per-role optimizer configuration (e.g., weight decay exemption for biases).
IReadOnlyList<OutputAxisContract> OutputAxesFor(int inputRank)
IReadOnlyList<Tensor<T>> GetTrainableParameters() Returns all trainable parameter tensors marked with [TrainableParameter].
void ResetState()
void SetTrainableParameters(IReadOnlyList<Tensor<T>> parameters) Replaces trainable parameter tensors (e.g., with ParameterBuffer views).
void UpdateParameters(T learningRate)
void ZeroGrad() Clears all gradient fields discovered by convention ({paramName}Gradient).