IQkvTransform<T>
Interfaces · AiDotNet.Interfaces
Transforms an attention block’s projected queries, keys and values BEFORE they are split into heads and scored.
Known implementations: UniVSTQkvTransform<T>
For Beginners
Attention turns each input into three tensors — a query, a key and a value — before deciding what to pay attention to. This lets a technique adjust those three first, which is how some style-transfer methods steer a frozen model without retraining it.
How It Works
This exists for methods that steer attention by rewriting Q/K/V rather than by rewriting the
block’s output. IAttentionBlockDecorator cannot express those: it only sees
PostProcess(innerOutput), i.e. the tensor produced AFTER the softmax and the output
projection. Applying a Q/K/V-shaped operation there would transform a different tensor and
silently implement a different method.
UniVST (arXiv:2410.20084) is the motivating case: it blends the query with a content query at every timestep and AdaIN-aligns the key/value against a style key/value on a schedule. Both act on the projections, so neither is reachable from the output.
Opt-in and shape-preserving. Attention layers consult this only when one is attached, so a
null transform leaves their behaviour bit-identical. Each method MUST return a tensor with the same
shape it was given — the caller reshapes the result into
[batch, heads, sequence, headDimension] immediately afterwards and a changed element count
makes that reshape fail.
Implementations should compose from Engine operations so the transform is recorded on the
caller’s gradient tape. Building a fresh tensor and writing into it by index severs the tape, which
leaves the projection weights with no gradient.
Methods
| Method | Summary |
|---|---|
Tensor<T> TransformKey(Tensor<T> key) |
Transforms the projected keys, shaped [batch * sequenceKV, embeddingDimension]. |
Tensor<T> TransformQuery(Tensor<T> query) |
Transforms the projected queries, shaped [batch * sequenceQ, embeddingDimension]. |
Tensor<T> TransformValue(Tensor<T> value) |
Transforms the projected values, shaped [batch * sequenceKV, embeddingDimension]. |