MultimodalEditMapperLayer<T>
Layers · AiDotNet.NeuralNetworks.Layers
Maps joint image/instruction hidden states to a fixed-length diffusion context using learned queries and an encoder-decoder transformer with true cross-attention.
Inheritance: LayerBase<T> → MultimodalEditMapperLayer<T>
Implements: IShapeContract
How It Works
The default geometry follows the released MGIE mapper: eight 4096-wide edit states, a 512-wide four-head transformer with four encoder and four decoder blocks, 77 learned queries, and a 768-wide output. This is a trainable native component, not a pretrained checkpoint loader. Its edit-token embeddings are both the appended MLLM input tokens and the residual embeddings added to their contextual hidden states before the mapper’s input projection.
Constructors
| Constructor | Summary |
|---|---|
MultimodalEditMapperLayer(int inputDim, int hiddenDim, int outputDim, int editTokenCount, int queryCount, int numHeads, int numLayers, double dropoutRate) |
Creates a registered, trainable fixed-query multimodal mapper. |
Properties
| Property | Summary |
|---|---|
bool SupportsTraining { get; } |
Methods
| Method | Summary |
|---|---|
Dictionary<string, string> GetParameterRoles() |
Returns parameter roles for per-role optimizer configuration (e.g., weight decay exemption for biases). |
IReadOnlyList<ILayer<T>> GetSubLayers() |
|
IReadOnlyList<OutputAxisContract> OutputAxesFor(int inputRank) |
|
IReadOnlyList<Tensor<T>> GetTrainableParameters() |
Returns all trainable parameter tensors marked with [TrainableParameter]. |
void ResetState() |
|
void SetTrainableParameters(IReadOnlyList<Tensor<T>> parameters) |
Replaces trainable parameter tensors (e.g., with ParameterBuffer views). |
void ZeroGrad() |
Clears all gradient fields discovered by convention ({paramName}Gradient). |