IQkvTransform<T>

Interfaces · AiDotNet.Interfaces

Transforms an attention block’s projected queries, keys and values BEFORE they are split into heads and scored.

Known implementations: UniVSTQkvTransform<T>

For Beginners

Attention turns each input into three tensors — a query, a key and a value — before deciding what to pay attention to. This lets a technique adjust those three first, which is how some style-transfer methods steer a frozen model without retraining it.

How It Works

This exists for methods that steer attention by rewriting Q/K/V rather than by rewriting the block’s output. IAttentionBlockDecorator cannot express those: it only sees PostProcess(innerOutput), i.e. the tensor produced AFTER the softmax and the output projection. Applying a Q/K/V-shaped operation there would transform a different tensor and silently implement a different method.

UniVST (arXiv:2410.20084) is the motivating case: it blends the query with a content query at every timestep and AdaIN-aligns the key/value against a style key/value on a schedule. Both act on the projections, so neither is reachable from the output.

Opt-in and shape-preserving. Attention layers consult this only when one is attached, so a null transform leaves their behaviour bit-identical. Each method MUST return a tensor with the same shape it was given — the caller reshapes the result into [batch, heads, sequence, headDimension] immediately afterwards and a changed element count makes that reshape fail.

Implementations should compose from Engine operations so the transform is recorded on the caller’s gradient tape. Building a fresh tensor and writing into it by index severs the tape, which leaves the projection weights with no gradient.

Methods

Method Summary
Tensor<T> TransformKey(Tensor<T> key) Transforms the projected keys, shaped [batch * sequenceKV, embeddingDimension].
Tensor<T> TransformQuery(Tensor<T> query) Transforms the projected queries, shaped [batch * sequenceQ, embeddingDimension].
Tensor<T> TransformValue(Tensor<T> value) Transforms the projected values, shaped [batch * sequenceKV, embeddingDimension].