MultimodalEditMapperLayer<T>

Layers · AiDotNet.NeuralNetworks.Layers

Maps joint image/instruction hidden states to a fixed-length diffusion context using learned queries and an encoder-decoder transformer with true cross-attention.

Inheritance: LayerBase<T> → MultimodalEditMapperLayer<T>

Implements: IShapeContract

How It Works

The default geometry follows the released MGIE mapper: eight 4096-wide edit states, a 512-wide four-head transformer with four encoder and four decoder blocks, 77 learned queries, and a 768-wide output. This is a trainable native component, not a pretrained checkpoint loader. Its edit-token embeddings are both the appended MLLM input tokens and the residual embeddings added to their contextual hidden states before the mapper’s input projection.

Constructors

Constructor Summary
MultimodalEditMapperLayer(int inputDim, int hiddenDim, int outputDim, int editTokenCount, int queryCount, int numHeads, int numLayers, double dropoutRate) Creates a registered, trainable fixed-query multimodal mapper.

Properties

Property Summary
bool SupportsTraining { get; }

Methods

Method Summary
Dictionary<string, string> GetParameterRoles() Returns parameter roles for per-role optimizer configuration (e.g., weight decay exemption for biases).
IReadOnlyList<ILayer<T>> GetSubLayers()
IReadOnlyList<OutputAxisContract> OutputAxesFor(int inputRank)
IReadOnlyList<Tensor<T>> GetTrainableParameters() Returns all trainable parameter tensors marked with [TrainableParameter].
void ResetState()
void SetTrainableParameters(IReadOnlyList<Tensor<T>> parameters) Replaces trainable parameter tensors (e.g., with ParameterBuffer views).
void ZeroGrad() Clears all gradient fields discovered by convention ({paramName}Gradient).