NoisePredictorBase<T>

Base Classes · AiDotNet.Diffusion.NoisePredictors

Base class for noise prediction networks used in diffusion models.

Implements: IModelShape, INoisePredictor<T>, IParameterLayoutSource, IParameterManifestProvider, IParameterSurfaceLifecycle

Known derived types: DiTNoisePredictor<T>, FlagDiTPredictor<T>, MMDiTNoisePredictor<T>, UNetNoisePredictor<T>, UViTNoisePredictor<T>, VideoUNetPredictor<T>

For Beginners

This is the foundation that all noise prediction networks build upon. Noise predictors are the neural networks at the heart of diffusion models that learn to predict what noise was added to a sample. Different architectures (U-Net, DiT, etc.) extend this base class.

How It Works

This abstract base class provides common functionality for all noise predictors, including timestep embedding, parameter management, serialization, and gradient computation.

Properties

Property Summary
ILossFunction<T> DefaultLossFunction { get; }
ParameterLayoutSnapshot ParameterLayout { get; }
bool ActivationCheckpointingEnabled { get; set; } Whether this predictor recomputes block activations during backward instead of storing them (activation checkpointing — the standard transformer memory/compute trade).
bool SupportsBatchedTimeEmbedding { get; } Whether PredictNoiseWithEmbedding accepts a [B, TimeEmbeddingDim] embedding, one row per batch element, and returns what PredictNoise returns for each element at its own timestep.
bool SupportsCFG { get; }
bool SupportsCrossAttention { get; }
bool SupportsParameterInitialization { get; }
int BaseChannels { get; }
int ContextDimension { get; }
int InputChannels { get; }
int OutputChannels { get; }
int TimeEmbeddingDim { get; }
long ParameterCount { get; }

Methods

Method Summary
Dictionary<Tensor<T>, Tensor<T>> ComputeGradientsWithTape(Tensor<T> input, Tensor<T> target, Tensor<T>[] trainableParams, Func<Tensor<T>, Tensor<T>, Tensor<T>> lossBuilder, Func<Tensor<T>, Tensor<T>> forwardBuilder) Computes gradients using the engine’s GradientTape for automatic differentiation.
Dictionary<string, T> GetFeatureImportance()
DynamicShapeInfo GetDynamicShapeInfo()
IEnumerable<Tensor<T>> GetParameterChunks() Streams the predictor’s trainable weight tensors per-tensor without materialising a flat aggregate, mirroring PyTorch’s nn.Module.parameters() generator pattern.
IEnumerable<int> GetActiveFeatureIndices()
IFullModel<T, Tensor<T>, Tensor<T>> DeepCopy()
IFullModel<T, Tensor<T>, Tensor<T>> WithParameters(Vector<T> parameters)
INoisePredictor<T> Clone() Creates a deep copy of the noise predictor.
IReadOnlyList<ParameterSlotDescriptor> GetParameterLayout()
ModelMetadata<T> GetModelMetadata()
Tensor<T> GetTimestepEmbedding(Tensor<T> timesteps) The sinusoidal timestep embedding of a [B] tensor of timesteps, [B, TimeEmbeddingDim], built from engine ops.
Tensor<T> GetTimestepEmbedding(int timestep) The sinusoidal timestep embedding of a [B] tensor of timesteps, [B, TimeEmbeddingDim], built from engine ops.
Tensor<T> Predict(Tensor<T> input)
Tensor<T> PredictNoise(Tensor<T> noisySample, int timestep, Tensor<T> conditioning)
Tensor<T> PredictNoiseBatched(Tensor<T> noisyBatch, int[] timesteps, Tensor<T> conditioning) Batched noise prediction (industry-standard: HuggingFace diffusers / DDPM reference sample one timestep per batch element and condition per-element).
Tensor<T> PredictNoiseWithEmbedding(Tensor<T> noisySample, Tensor<T> timeEmbedding, Tensor<T> conditioning)
ValueTask<Tensor<T>> PredictNoiseAsync(Tensor<T> noisySample, int timestep, Tensor<T> conditioning, CancellationToken cancellationToken) Async overload of PredictNoise.
Vector<T> ComputeGradients(Tensor<T> input, Tensor<T> target, ILossFunction<T> lossFunction)
Vector<T> GetParameters()
Vector<T> SanitizeParameters(Vector<T> parameters)
bool IsFeatureUsed(int featureIndex)
byte[] Serialize()
int[] GetInputShape()
int[] GetOutputShape()
void ApplyGradients(Vector<T> gradients, T learningRate)
void Deserialize(byte[] data)
void Dispose()
void LoadModel(string filePath)
void LoadState(Stream stream)
void SaveModel(string filePath)
void SaveState(Stream stream)
void SetActiveFeatureIndices(IEnumerable<int> featureIndices)
void SetParameterChunks(IEnumerable<Tensor<T>> chunks) Streaming counterpart to SetParameters: assigns the predictor’s weights from per-tensor chunks in GetParameterChunks order without materializing a flat aggregate.
void SetParameters(Vector<T> parameters)
void Train(Tensor<T> input, Tensor<T> expectedOutput)