TimeEmbeddingLayer<T>
Layers · AiDotNet.NeuralNetworks.Layers
Represents a time embedding layer that encodes timesteps using sinusoidal embeddings for diffusion models.
Inheritance: LayerBase<T> → TimeEmbeddingLayer<T>
Implements: IShapeContract
For Beginners
In diffusion models, the network needs to know “what time step are we at?”
- At early timesteps (t near 0), images are clean and noise is minimal
- At late timesteps (t near T), images are mostly noise
- The network needs this information to know how much denoising to apply
This layer encodes the timestep number into a rich vector representation that:
- Uses sine and cosine functions at different frequencies (sinusoidal encoding)
- Passes through a small neural network (MLP) to learn task-specific representations
- Gets injected into every ResNet block of the U-Net
The sinusoidal encoding is inspired by transformer positional encodings:
- Low frequencies capture coarse time information
- High frequencies capture fine-grained time details
How It Works
The time embedding layer converts scalar timesteps into high-dimensional embeddings using sinusoidal functions, similar to positional encodings in transformers. This embedding is then projected through a small MLP to produce the final time conditioning vector used in diffusion U-Net blocks.
Constructors
| Constructor | Summary |
|---|---|
TimeEmbeddingLayer(int embeddingDim, int outputDim, int maxTimestep) |
Initializes a new instance of the TimeEmbeddingLayer class. |
Properties
| Property | Summary |
|---|---|
bool SupportsTraining { get; } |
Methods
| Method | Summary |
|---|---|
Dictionary<string, string> GetParameterRoles() |
Returns parameter roles for per-role optimizer configuration (e.g., weight decay exemption for biases). |
IReadOnlyList<OutputAxisContract> OutputAxesFor(int inputRank) |
|
IReadOnlyList<Tensor<T>> GetTrainableParameters() |
Returns all trainable parameter tensors marked with [TrainableParameter]. |
void ClearGradients() |
|
void ResetState() |
Resets the internal state of the layer. |
void SetTrainableParameters(IReadOnlyList<Tensor<T>> parameters) |
Replaces trainable parameter tensors (e.g., with ParameterBuffer views). |
void UpdateParameters(T learningRate) |
Updates the parameters of the layer using the calculated gradients. |
void ZeroGrad() |
Clears all gradient fields discovered by convention ({paramName}Gradient). |