CompiledStepRandom<T>

Helpers & Utilities · AiDotNet.Training

Random tensors a training step draws fresh on every step (reparameterization noise, a GAN’s latent batch, a diffusion step’s noise), declared so that a compiled training plan redraws them on every replay.

For Beginners

if a training forward needs fresh random numbers each step, get them from here and the fast compiled training path stays correct.

How It Works

The fused training path traces the model’s forward and loss once and then replays the compiled graph. A tensor the forward builds on the host is captured by reference as a constant of that graph, and the closure that built it does not run again on a replay. Noise drawn with new Tensor and a host loop therefore stays the first step’s draw for the whole run, and the model trains on one fixed sample: a VAE whose epsilon never changes, a generator that only ever sees one latent batch. Nothing fails; the loss falls on the wrong objective.

Drawing through this type fixes that for every model at once. On the eager tape it is an ordinary draw. While a plan is being traced, each draw is recorded with the plan, and before every later replay the plan’s draws are refilled in place from the same generator: the same mechanism dropout masks use, generalized so that a model does not need a layer type to own its randomness.

Methods

Method Summary
Tensor<T> Draw(int[] shape, Action<Tensor<T>> fill) A per-step random tensor of shape, filled by fill: on the eager tape an ordinary draw; while a plan is being traced, fill is recorded and refills the tensor in place before every later replay.
Tensor<T> StandardNormal(int[] shape, Random random) A tensor of independent standard-normal samples, redrawn on every compiled replay.
Tensor<T> Uniform(int[] shape, Random random, double min, double max) A tensor of independent samples uniform on [min, max), redrawn on every compiled replay.