TLoRAAttentionAdapter<T>
Models & Types · AiDotNet.Diffusion.StyleTransfer
Wraps a UNet attention block and adds T-LoRA’s timestep-masked low-rank update to its output.
Inheritance: LayerBase<T> → TLoRAAttentionAdapter<T>
Implements: IAttentionBlockDecorator<T>, IShapeContract
For Beginners
This sits on top of an existing attention block and adds a small, learnable correction to what it produces. How much freedom that correction has depends on how noisy the current generation step is — that is the whole idea of T-LoRA.
How It Works
This is the injection site for Soboleva et al., “T-LoRA: Single Image Diffusion Model
Customization Without Overfitting” (arXiv:2507.05964). The paper adapts the self- and
cross-attention projections of the UNet; this decorator adapts the block’s OUTPUT projection
position, which is the standard to_out LoRA target, and leaves the wrapped block’s own
weights untouched and frozen.
Why a decorator rather than editing the attention layers.DiffusionAttention holds no q/k/v matrices of its
own — it delegates to FlashAttentionLayer or MultiHeadAttentionLayer, which every
other diffusion model in the library also uses. Reaching into those to add per-timestep masking
would put T-LoRA’s mechanism on the hot path of models that do not want it. Wrapping keeps the
change local to the one model that asked for it, and keeps the base weights frozen by
construction rather than by convention: this layer reports only the adapter’s parameters, so an
optimizer walking it cannot move W.
The timestep.Forward carries no timestep, and the rank mask is a
function of it, so the value is supplied out-of-band via CurrentTimestep — set once
per denoising step by the owning model before it walks the network. That is deliberate rather than
convenient: threading a timestep through every layer signature would change a library-wide
interface for the benefit of one model.
Constructors
| Constructor | Summary |
|---|---|
TLoRAAttentionAdapter(ILayer<T> inner, int channels, int rank, int totalTimesteps, Random random) |
Wraps inner with a T-LoRA adapter over channels width. |
Properties
| Property | Summary |
|---|---|
ILayer<T> Inner { get; } |
Gets the wrapped attention block, whose own weights this adapter never modifies. |
TimestepDependentLora<T> Adapter { get; } |
Gets the timestep-dependent low-rank adapter applied to the wrapped block’s output. |
bool SupportsTraining { get; } |
|
int AdapterParameterCount { get; } |
Gets the number of adapter parameters (A, B and S), excluding the wrapped block’s. |
int CurrentTimestep { get; set; } |
Gets or sets the diffusion timestep the rank mask is evaluated at. |
Methods
| Method | Summary |
|---|---|
IReadOnlyList<ILayer<T>> GetSubLayers() |
|
IReadOnlyList<OutputAxisContract> OutputAxesFor(int inputRank) |
|
Tensor<T> PostProcess(Tensor<T> output) |
Adds the timestep-masked low-rank update to whatever the wrapped block produced. |
Vector<T> GetAdapterState() |
The adapter’s own trainable state — A, then B, then S — kept OUT of GetParameters on purpose. |
void ResetState() |
|
void SetAdapterState(Vector<T> state) |
Restores state produced by GetAdapterState. |