HouseholderSubspaceLayer<T>
Layers · AiDotNet.NeuralNetworks.Layers
Matrix-free orthogonal projection y = x Q (or y = x Qᵀ) parameterized as a product of
k Householder reflections Q = H₀ H₁ ⋯ H_{k−1}, where each
Hⱼ = I − (2 / (‖uⱼ‖₂² + ε)) uⱼ uⱼᵀ uses only O(k · D) parameters and 4 k · M · D FLOPs
without ever materializing a D × D matrix.
Inheritance: LayerBase<T> → HouseholderSubspaceLayer<T>
Implements: IShapeContract
How It Works
Every non-zero reflector vector uⱼ ∈ ℝᴰ defines an exact symmetric orthogonal reflection
(Hⱼᵀ = Hⱼ, Hⱼ² = I). Consequently Qᵀ Q = I holds by construction for arbitrary unconstrained
gradient updates to uⱼ without SVD, QR retraction, or Cayley matrix inversions, preserving activation
and gradient Euclidean norms across arbitrarily deep shared-parameter stacks (e.g. 100+ layers).
Setting Transpose to true applies the reflections in reverse order
(H_{k−1} ⋯ H₁ H₀ = Qᵀ = Q⁻¹), providing the exact inverse/transpose rotation.
Constructors
| Constructor | Summary |
|---|---|
HouseholderSubspaceLayer(int dimension, int numReflectors, bool transpose) |
Creates a matrix-free orthogonal Householder subspace rotation layer of width dimension using numReflectors reflections. |
Properties
| Property | Summary |
|---|---|
Tensor<T> Reflectors { get; } |
Gets the underlying [NumReflectors, Dimension] trainable reflector matrix. |
bool SupportsTraining { get; } |
|
bool Transpose { get; } |
Gets whether the reflection chain is applied in reverse order (Qᵀ = Q⁻¹). |
int Dimension { get; } |
Gets the feature dimension D rotated by this layer. |
int NumReflectors { get; } |
Gets the number of Householder reflections k composed by this layer. |
Methods
| Method | Summary |
|---|---|
Dictionary<string, string> GetParameterRoles() |
Returns parameter roles for per-role optimizer configuration (e.g., weight decay exemption for biases). |
IReadOnlyList<OutputAxisContract> OutputAxesFor(int inputRank) |
|
IReadOnlyList<Tensor<T>> GetTrainableParameters() |
Returns all trainable parameter tensors marked with [TrainableParameter]. |
void ResetState() |
|
void SetTrainableParameters(IReadOnlyList<Tensor<T>> parameters) |
Replaces trainable parameter tensors (e.g., with ParameterBuffer views). |
void UpdateParameters(T learningRate) |
|
void ZeroGrad() |
Clears all gradient fields discovered by convention ({paramName}Gradient). |