UNetNoisePredictor<T>
Models & Types · AiDotNet.Diffusion.NoisePredictors
U-Net architecture for noise prediction in diffusion models.
Inheritance: NoisePredictorBase<T> → UNetNoisePredictor<T>
For Beginners
Think of U-Net like a funnel:
- Encoder (going down): Compresses the image, capturing patterns at different scales
- Middle: Processes the most compressed representation
- Decoder (going up): Expands back to original size, using skip connections
Skip connections are like “shortcuts” that connect encoder layers directly to decoder layers, helping the network preserve fine details that might otherwise be lost during compression.
How It Works
The U-Net is the most common architecture for diffusion model noise prediction. It has an encoder-decoder structure with skip connections that help preserve fine-grained details during the denoising process.
This implementation follows the Stable Diffusion architecture with:
- Residual blocks with group normalization
- Self-attention at lower resolutions
- Cross-attention for text conditioning
- Time embedding injection via adaptive normalization
Example
var predictor = new UNetNoisePredictor<float>(inputChannels: 4, baseChannels: 320, contextDim: 768);
var noisyLatent = Tensor<float>.CreateRandom(new[] { 1, 4, 64, 64 });
var predicted = predictor.PredictNoise(noisyLatent, timestep: 500);
Constructors
| Constructor | Summary |
|---|---|
UNetNoisePredictor(NeuralNetworkArchitecture<T> architecture, int inputChannels, int? outputChannels, int baseChannels, int[] channelMultipliers, int numResBlocks, int[] attentionResolutions, int contextDim, int numHeads, int inputHeight, List<UNetBlock<T>> encoderBlocks, List<UNetBlock<T>> middleBlocks, List<UNetBlock<T>> decoderBlocks, ILossFunction<T> lossFunction, int? seed) |
Initializes a new instance of the UNetNoisePredictor class with full customization support. |
Properties
| Property | Summary |
|---|---|
bool SupportsBatchedTimeEmbedding { get; } |
|
bool SupportsCFG { get; } |
|
bool SupportsCrossAttention { get; } |
|
int BaseChannels { get; } |
|
int ContextDimension { get; } |
|
int InputChannels { get; } |
|
int OutputChannels { get; } |
|
int TimeEmbeddingDim { get; } |
|
int[] ChannelMultipliers { get; } |
Gets the per-level channel multipliers of this UNet’s encoder/decoder. |
Methods
| Method | Summary |
|---|---|
Tensor<T> PredictNoise(Tensor<T> noisySample, int timestep, Tensor<T> conditioning) |
|
Tensor<T> PredictNoiseWithEmbedding(Tensor<T> noisySample, Tensor<T> timeEmbedding, Tensor<T> conditioning) |
|
bool CompileForward(Tensor<T> sampleNoisy, int sampleTimestep, Tensor<T> conditioning) |
Eagerly compiles the UNet forward pass for the given sample input shape, storing the plan in the per-instance cache. |
int DecorateAttentionBlocks(Func<ILayer<T>, int, bool, ILayer<T>> decorator) |
Replaces every self- and cross-attention block with the result of decorator, so an adapter (LoRA and friends) can be injected without this class knowing about it. |