DiffusionAttentionLayer<T>

Layers ยท AiDotNet.Diffusion.NoisePredictors

Lazy multi-head attention with distinct query and context dimensions.

Inheritance: LayerBase<T> โ†’ DiffusionAttentionLayer<T>

Implements: IShapeContract

How It Works

This is the projection contract required by diffusion cross attention: Q maps from the U-Net channel width while K/V map from the text-encoder width. All math uses engine operations, including fused scaled-dot-product attention, so the same graph is tape-connected on CPU and every GPU backend.

Constructors

Constructor Summary
DiffusionAttentionLayer(int queryDimension, int contextDimension, int headCount, bool zeroOutputProjection) Creates self- or cross-attention without eager weight allocation.

Properties

Property Summary
bool IsOutputProjectionZeroInitialized { get; } Gets whether the output projection starts at zero.
bool SupportsTraining { get; }
int ContextDimension { get; } Gets the key/value context feature width.
int HeadCount { get; } Gets the number of attention heads.
int QueryDimension { get; } Gets the query/output feature width.

Methods

Method Summary
Dictionary<string, string> GetParameterRoles() Returns parameter roles for per-role optimizer configuration (e.g., weight decay exemption for biases).
IReadOnlyList<OutputAxisContract> OutputAxesFor(int inputRank)
IReadOnlyList<Tensor<T>> GetTrainableParameters() Returns all trainable parameter tensors marked with [TrainableParameter].
Tensor<T> Forward(Tensor<T> query, Tensor<T> context) Runs attention with query [B,S,Q] and context [B,N,C].
void ResetState()
void SetTrainableParameters(IReadOnlyList<Tensor<T>> parameters) Replaces trainable parameter tensors (e.g., with ParameterBuffer views).
void UpdateParameters(T learningRate)
void ZeroGrad() Clears all gradient fields discovered by convention ({paramName}Gradient).