TripleTextConditioner<T>
Models & Types · AiDotNet.Diffusion.Conditioning
Triple text encoder conditioning module combining two CLIP encoders and a T5 encoder.
Inheritance: CompositeConditioningBase<T> → TripleTextConditioner<T>
For Beginners
This uses THREE text encoders together for the best possible results.
Why three encoders?
- CLIP ViT-L/14 (768-dim): Fast, general visual understanding of your prompt
- OpenCLIP ViT-bigG/14 (1280-dim): Larger, more detailed visual understanding
- T5-XXL (4096-dim): Deep language understanding for complex prompts
How they work together:
- Both CLIP encoders produce “pooled” embeddings (one vector each summarizing the prompt)
→ These are concatenated into a combined 2048-dim (768+1280) vector → Used for the global conditioning vector fed to the MMDiT timestep embedder
- T5 produces “sequence” embeddings (one vector per token, 4096-dim each)
→ Used for cross-attention, giving fine-grained control over details
Used by:
- Stable Diffusion 3 (SD3): All three encoders
- SD3 Turbo: All three encoders with fewer steps
How It Works
Stable Diffusion 3 uses three text encoders to achieve the highest quality text understanding: two CLIP encoders for visual-semantic alignment (with pooled embeddings) and a T5 encoder for detailed language understanding (with cross-attention sequence embeddings).
Reference: Esser et al., “Scaling Rectified Flow Transformers for High-Resolution Image Synthesis”, ICML 2024
Constructors
| Constructor | Summary |
|---|---|
TripleTextConditioner(CLIPTextConditioner<T> clipLEncoder, CLIPTextConditioner<T> clipGEncoder, T5TextConditioner<T> t5Encoder) |
Initializes a new triple text encoder conditioning module for SD3. |
Properties
| Property | Summary |
|---|---|
ConditioningType ConditioningType { get; } |
|
bool ProducesPooledOutput { get; } |
Gets whether this module produces pooled output (yes, from both CLIP encoders). |
int CLIPGEmbeddingDimension { get; } |
Gets the second CLIP/OpenCLIP encoder’s embedding dimension (1280 for ViT-bigG/14). |
int CLIPLEmbeddingDimension { get; } |
Gets the first CLIP encoder’s embedding dimension (768 for ViT-L/14). |
int CombinedPooledDimension { get; } |
Gets the combined pooled embedding dimension (CLIP-L + CLIP-G = 768 + 1280 = 2048). |
int EmbeddingDimension { get; } |
Gets the T5 context dimension for cross-attention (4096 for T5-XXL). |
int MaxSequenceLength { get; } |
Gets the maximum sequence length (uses T5’s longer sequence length). |
int T5EmbeddingDimension { get; } |
Gets the T5 encoder’s embedding dimension (4096 for T5-XXL). |
Methods
| Method | Summary |
|---|---|
Tensor<T> Encode(Tensor<T> input) |
|
Tensor<T> EncodeText(Tensor<T> tokenIds, Tensor<T> attentionMask) |
|
Tensor<T> GetCombinedPooledEmbedding(string text) |
Gets the combined pooled embedding from both CLIP encoders for the given text. |
Tensor<T> GetPooledEmbedding(Tensor<T> sequenceEmbeddings) |
|
Tensor<T> GetUnconditionalEmbedding(int batchSize) |
|
Tensor<T> Tokenize(string text) |
|
Tensor<T> TokenizeBatch(string[] texts) |
|
TripleTextConditioner<T> FromPretrained(CLIPVariant clipLVariant, CLIPVariant clipGVariant, T5Variant t5Variant, string cacheDir) |
Loads a paper-canonical SDXL-style triple encoder with its real pretrained tokenizers from HuggingFace (CLIP-L + CLIP-G + T5). |
ValueTuple<Tensor<T>, Tensor<T>> EncodeTriple(string text) |
Encodes text using all three encoders (CLIP-L, CLIP-G/OpenCLIP, and T5). |
ValueTuple<Tensor<T>, Tensor<T>> GetUnconditionalTriple(int batchSize) |
Gets unconditional embeddings from all three encoders. |