InstantStyleModel<T>
Models & Types · AiDotNet.Diffusion.StyleTransfer
InstantStyle model for zero-shot style transfer using IP-Adapter with style-content separation.
Inheritance: DiffusionModelBase<T> → LatentDiffusionModelBase<T> → InstantStyleModel<T>
For Beginners
InstantStyle transfers artistic styles instantly — no training or fine-tuning needed per style. It smartly injects style information into specific parts of the model to capture the look without copying the content of the style image.
How It Works
InstantStyle achieves high-quality style transfer in a single forward pass by selectively injecting IP-Adapter image features into only the style-relevant attention layers, preventing content leakage from the style reference.
Reference: Wang et al., “InstantStyle: Free Lunch towards Style-Preserving in Text-to-Image Generation”, 2024
Example
var options = new DiffusionModelOptions<float> { LatentChannels = 4, DefaultInferenceSteps = 30 };
var styleRef = Tensor<float>.CreateRandom(new[] { 1, 4, 64, 64 });
var trainX = Tensor<float>.CreateRandom(4, 8);
var trainY = Tensor<float>.CreateRandom(4, 2);
var result = new AiModelBuilder<float, Tensor<float>, Tensor<float>>()
.ConfigureModel(new InstantStyleModel<float>(options: options))
.Build(trainX, trainY);
var stylized = result.Predict(styleRef);
Constructors
| Constructor | Summary |
|---|---|
InstantStyleModel(NeuralNetworkArchitecture<T> architecture, DiffusionModelOptions<T> options, INoiseScheduler<T> scheduler, UNetNoisePredictor<T> predictor, StandardVAE<T> vae, IConditioningModule<T> conditioner, int? seed) |
Properties
| Property | Summary |
|---|---|
IConditioningModule<T> Conditioner { get; } |
|
INoisePredictor<T> NoisePredictor { get; } |
|
IReadOnlyList<int> InjectionBlocks { get; } |
The blocks that receive reference image features, 1-based and ascending. |
IVAEModel<T> VAE { get; } |
|
bool InjectLayoutBlock { get; set; } |
Whether reference features are also injected into the layout block (LayoutBlockIndex). |
int LatentChannels { get; } |
Methods
| Method | Summary |
|---|---|
ModelMetadata<T> GetModelMetadata() |
|
Tensor<T> DecoupleStyleFromContent(Tensor<T> referenceImageFeatures, Tensor<T> contentTextFeatures) |
Decouples style from content by subtracting content-text features from reference-image features in the shared CLIP space. |
bool IsInjectionBlock(int oneBasedBlockIndex) |
Whether block oneBasedBlockIndex receives reference image features. |
double[] BuildBlockInjectionScales() |
Per-block injection scale for the whole UNet: 1 at an injection block, 0 everywhere else. |
Fields
| Field | Summary |
|---|---|
int LayoutBlockIndex |
The LAYOUT block, 1-based. |
int StyleBlockIndex |
The STYLE block, 1-based within TransformerBlockCount. |
int TransformerBlockCount |
Number of transformer blocks in the SDXL UNet: “There are 11 transformer blocks with SDXL, 4 for downsample blocks, 1 for middle block, 6 for upsample blocks.” |