RewardGuidance<T>

Models & Types · AiDotNet.Diffusion.Alignment

Reward-guided sampling for inference-time alignment of diffusion models.

For Beginners

Instead of permanently changing the model (like RLHF does), reward guidance acts like a compass during image generation. At each denoising step, it asks the reward model “which direction leads to better images?” and nudges the generation that way. This is flexible — you can change the reward model or guidance strength without retraining.

How It Works

Reward guidance modifies the sampling process at inference time by incorporating gradients from a reward model. Unlike RLHF which fine-tunes the model, reward guidance keeps the base model frozen and steers the denoising trajectory toward higher-reward regions using the reward model’s gradient signal.

Reference: Xu et al., “Imagereward: Learning and evaluating human preferences for text-to-image generation”, NeurIPS 2023

Constructors

Constructor Summary
RewardGuidance(double guidanceScale, double truncationTimestep, double gradientClipNorm) Initializes a new reward guidance module.

Properties

Property Summary
double GradientClipNorm { get; } Gets the gradient clipping norm.
double GuidanceScale { get; } Gets the guidance scale.
double TruncationTimestep { get; } Gets the truncation timestep fraction.

Methods

Method Summary
Vector<T> ApplyGuidance(Vector<T> noisePrediction, Vector<T> rewardGradient, double currentTimestepFraction) Applies reward guidance to a noise prediction by incorporating the reward gradient.