DualTextConditioner<T>

Models & Types · AiDotNet.Diffusion.Conditioning

Dual text encoder conditioning module combining CLIP and T5 encoders.

Implements: IConditioningModule<T>

For Beginners

This uses TWO text encoders together for better results.

Why two encoders?

  • CLIP: Great at understanding the visual “gist” of your prompt

(e.g., knowing what a cat looks like, what “golden light” means visually)

  • T5: Great at understanding language details

(e.g., counting objects, understanding spatial relationships, rendering text)

How they work together:

  1. CLIP produces a “pooled” embedding (one vector summarizing the whole prompt)

→ Used for the global style/content of the image

  1. T5 produces “sequence” embeddings (one vector per token)

→ Used for cross-attention, giving fine-grained control

Used by:

  • FLUX.1: CLIP ViT-L/14 (768-dim) + T5-XXL (4096-dim)
  • SD3: CLIP ViT-L/14 (768-dim) + OpenCLIP ViT-bigG/14 (1280-dim) + T5-XXL (4096-dim)
  • Imagen: T5-XXL (text only, no CLIP)

How It Works

Many modern diffusion models use multiple text encoders together to get the best of each encoder’s strengths. This module combines a CLIP encoder (for pooled embeddings and visual-semantic alignment) with a T5 encoder (for detailed text understanding and cross-attention).

Constructors

Constructor Summary
DualTextConditioner(CLIPTextConditioner<T> clipEncoder, T5TextConditioner<T> t5Encoder) Initializes a new dual text encoder conditioning module.

Properties

Property Summary
ConditioningType ConditioningType { get; }
bool ProducesPooledOutput { get; } Gets whether this module produces pooled output (yes, from CLIP).
int CLIPEmbeddingDimension { get; } Gets the CLIP encoder’s embedding dimension (for pooled embeddings).
int EmbeddingDimension { get; } Gets the combined context dimension (T5 embedding dimension for cross-attention).
int MaxSequenceLength { get; } Gets the maximum sequence length (uses T5’s longer sequence length).
int T5EmbeddingDimension { get; } Gets the T5 encoder’s embedding dimension (for cross-attention).

Methods

Method Summary
DualTextConditioner<T> FromPretrained(CLIPVariant clipVariant, T5Variant t5Variant, string cacheDir) Loads a paper-canonical SD3-style dual encoder with its real pretrained tokenizers from HuggingFace (CLIP + T5).
Tensor<T> Encode(Tensor<T> input)
Tensor<T> EncodeText(Tensor<T> tokenIds, Tensor<T> attentionMask)
Tensor<T> GetPooledEmbedding(Tensor<T> sequenceEmbeddings)
Tensor<T> GetUnconditionalEmbedding(int batchSize)
Tensor<T> Tokenize(string text)
Tensor<T> TokenizeBatch(string[] texts)
ValueTuple<Tensor<T>, Tensor<T>> EncodeDual(string text) Encodes text using both CLIP and T5 encoders.
ValueTuple<Tensor<T>, Tensor<T>> GetUnconditionalDual(int batchSize) Gets unconditional embeddings from both encoders.