T2IAdapterModel<T>
Models & Types · AiDotNet.Diffusion.Control
T2I-Adapter model for adding spatial control to text-to-image diffusion models.
Inheritance: DiffusionModelBase<T> → LatentDiffusionModelBase<T> → T2IAdapterModel<T>
For Beginners
T2I-Adapter adds structural guidance to image generation:
How T2I-Adapter works:
- A spatial condition (depth map, sketch, pose, etc.) is processed by the adapter network
- The adapter produces multi-scale feature maps matching the U-Net encoder stages
- These features are added to the U-Net during denoising for structural guidance
- The base SD model generates images following both text AND spatial conditions
Key characteristics:
- Lightweight: Only ~77M parameters (vs ~860M for ControlNet)
- Pluggable: Works with any SD 1.5 or SDXL base model
- Composable: Multiple adapters can be combined (e.g., depth + sketch)
- No base model modification: Adapter weights are separate
- Fast training: Much faster to train than ControlNet
Supported condition types:
- Depth maps (MiDaS, ZoeDepth)
- Canny edge maps
- Sketch/line art
- OpenPose skeleton
- Semantic segmentation
- Color palette
Advantages over ControlNet:
- 10x fewer parameters
- Faster training and inference
- Composable (stack multiple adapters)
- Smaller model files
Limitations:
- Less precise control than ControlNet
- May not preserve fine spatial details as well
How It Works
T2I-Adapter is a lightweight adapter architecture that adds spatial conditioning (depth maps, sketches, pose, segmentation, etc.) to pre-trained text-to-image models without modifying the base model weights.
Technical specifications:
- Architecture: Lightweight encoder adapter with multi-scale feature injection
- Adapter: ~77M parameters, 4 downsampling stages matching U-Net encoder
- Compatible base models: SD 1.5 (768-dim), SDXL (varies)
- Input conditions: Any spatial map (depth, edge, pose, etc.)
- Adapter scale: Adjustable strength [0.0, 1.0] for blending
- Adapter channels: [320, 640, 1280, 1280] matching SD U-Net
Reference: Mou et al., “T2I-Adapter: Learning Adapters to Dig out More Controllable Ability for Text-to-Image Diffusion Models”, AAAI 2024
Example
// Create T2I-Adapter with SD 1.5 as base
var adapter = new T2IAdapterModel<float>();
// Generate a 512x512 image with depth-guided control
var image = adapter.GenerateFromText(
prompt: "A house on a hill at sunset",
negativePrompt: "blurry, low quality",
width: 512,
height: 512,
numInferenceSteps: 30,
guidanceScale: 7.5,
seed: 42);
Constructors
| Constructor | Summary |
|---|---|
T2IAdapterModel(NeuralNetworkArchitecture<T> architecture, DiffusionModelOptions<T> options, INoiseScheduler<T> scheduler, UNetNoisePredictor<T> unet, UNetNoisePredictor<T> adapterNetwork, StandardVAE<T> vae, IConditioningModule<T> conditioner, double adapterScale, int? seed) |
Initializes a new instance of T2IAdapterModel with full customization support. |
Properties
| Property | Summary |
|---|---|
IConditioningModule<T> Conditioner { get; } |
|
INoisePredictor<T> AdapterNetwork { get; } |
Gets the adapter network that processes spatial conditions. |
INoisePredictor<T> NoisePredictor { get; } |
|
IVAEModel<T> VAE { get; } |
|
double AdapterScale { get; } |
Gets the adapter conditioning scale [0.0, 1.0]. |
int LatentChannels { get; } |
Methods
| Method | Summary |
|---|---|
ModelMetadata<T> GetModelMetadata() |
|
Tensor<T> GenerateFromText(string prompt, string negativePrompt, int width, int height, int numInferenceSteps, double? guidanceScale, int? seed) |
|
Tensor<T> ImageToImage(Tensor<T> inputImage, string prompt, string negativePrompt, double strength, int numInferenceSteps, double? guidanceScale, int? seed) |
Fields
| Field | Summary |
|---|---|
int DefaultHeight |
Default image height for T2I-Adapter (matches SD 1.5). |
int DefaultWidth |
Default image width for T2I-Adapter (matches SD 1.5). |