DualTextConditioner<T>
Models & Types · AiDotNet.Diffusion.Conditioning
Dual text encoder conditioning module combining CLIP and T5 encoders.
Implements: IConditioningModule<T>
For Beginners
This uses TWO text encoders together for better results.
Why two encoders?
- CLIP: Great at understanding the visual “gist” of your prompt
(e.g., knowing what a cat looks like, what “golden light” means visually)
- T5: Great at understanding language details
(e.g., counting objects, understanding spatial relationships, rendering text)
How they work together:
- CLIP produces a “pooled” embedding (one vector summarizing the whole prompt)
→ Used for the global style/content of the image
- T5 produces “sequence” embeddings (one vector per token)
→ Used for cross-attention, giving fine-grained control
Used by:
- FLUX.1: CLIP ViT-L/14 (768-dim) + T5-XXL (4096-dim)
- SD3: CLIP ViT-L/14 (768-dim) + OpenCLIP ViT-bigG/14 (1280-dim) + T5-XXL (4096-dim)
- Imagen: T5-XXL (text only, no CLIP)
How It Works
Many modern diffusion models use multiple text encoders together to get the best of each encoder’s strengths. This module combines a CLIP encoder (for pooled embeddings and visual-semantic alignment) with a T5 encoder (for detailed text understanding and cross-attention).
Constructors
| Constructor | Summary |
|---|---|
DualTextConditioner(CLIPTextConditioner<T> clipEncoder, T5TextConditioner<T> t5Encoder) |
Initializes a new dual text encoder conditioning module. |
Properties
| Property | Summary |
|---|---|
ConditioningType ConditioningType { get; } |
|
bool ProducesPooledOutput { get; } |
Gets whether this module produces pooled output (yes, from CLIP). |
int CLIPEmbeddingDimension { get; } |
Gets the CLIP encoder’s embedding dimension (for pooled embeddings). |
int EmbeddingDimension { get; } |
Gets the combined context dimension (T5 embedding dimension for cross-attention). |
int MaxSequenceLength { get; } |
Gets the maximum sequence length (uses T5’s longer sequence length). |
int T5EmbeddingDimension { get; } |
Gets the T5 encoder’s embedding dimension (for cross-attention). |
Methods
| Method | Summary |
|---|---|
DualTextConditioner<T> FromPretrained(CLIPVariant clipVariant, T5Variant t5Variant, string cacheDir) |
Loads a paper-canonical SD3-style dual encoder with its real pretrained tokenizers from HuggingFace (CLIP + T5). |
Tensor<T> Encode(Tensor<T> input) |
|
Tensor<T> EncodeText(Tensor<T> tokenIds, Tensor<T> attentionMask) |
|
Tensor<T> GetPooledEmbedding(Tensor<T> sequenceEmbeddings) |
|
Tensor<T> GetUnconditionalEmbedding(int batchSize) |
|
Tensor<T> Tokenize(string text) |
|
Tensor<T> TokenizeBatch(string[] texts) |
|
ValueTuple<Tensor<T>, Tensor<T>> EncodeDual(string text) |
Encodes text using both CLIP and T5 encoders. |
ValueTuple<Tensor<T>, Tensor<T>> GetUnconditionalDual(int batchSize) |
Gets unconditional embeddings from both encoders. |