DiffusionAttentionLayer<T>
Layers ยท AiDotNet.Diffusion.NoisePredictors
Lazy multi-head attention with distinct query and context dimensions.
Inheritance: LayerBase<T> โ DiffusionAttentionLayer<T>
Implements: IShapeContract
How It Works
This is the projection contract required by diffusion cross attention: Q maps from the U-Net channel width while K/V map from the text-encoder width. All math uses engine operations, including fused scaled-dot-product attention, so the same graph is tape-connected on CPU and every GPU backend.
Constructors
| Constructor | Summary |
|---|---|
DiffusionAttentionLayer(int queryDimension, int contextDimension, int headCount, bool zeroOutputProjection) |
Creates self- or cross-attention without eager weight allocation. |
Properties
| Property | Summary |
|---|---|
bool IsOutputProjectionZeroInitialized { get; } |
Gets whether the output projection starts at zero. |
bool SupportsTraining { get; } |
|
int ContextDimension { get; } |
Gets the key/value context feature width. |
int HeadCount { get; } |
Gets the number of attention heads. |
int QueryDimension { get; } |
Gets the query/output feature width. |
Methods
| Method | Summary |
|---|---|
Dictionary<string, string> GetParameterRoles() |
Returns parameter roles for per-role optimizer configuration (e.g., weight decay exemption for biases). |
IReadOnlyList<OutputAxisContract> OutputAxesFor(int inputRank) |
|
IReadOnlyList<Tensor<T>> GetTrainableParameters() |
Returns all trainable parameter tensors marked with [TrainableParameter]. |
Tensor<T> Forward(Tensor<T> query, Tensor<T> context) |
Runs attention with query [B,S,Q] and context [B,N,C]. |
void ResetState() |
|
void SetTrainableParameters(IReadOnlyList<Tensor<T>> parameters) |
Replaces trainable parameter tensors (e.g., with ParameterBuffer views). |
void UpdateParameters(T learningRate) |
|
void ZeroGrad() |
Clears all gradient fields discovered by convention ({paramName}Gradient). |