MbPAOutputDistribution

Enums · AiDotNet.Enums

Which likelihood MbPA’s output head models, i.e. what log p(v | h, theta_x) means.

How It Works

This enum was referenced from four files and declared in none of them.MbPAOptions.OutputDistribution, MbPAOutputNetwork and LFTAlgorithm all named it, so the type was clearly intended; it simply never got written, and the whole project failed to compile on twelve CS0246s pointing at a name that existed nowhere in the tree.

The two members are not an arbitrary pair. They are exactly the cases for which a linear head has the SAME closed-form gradient, w (prediction - target) (x) h — softmax composed with cross entropy, and a unit-variance Gaussian composed with squared error. MbPA’s local adaptation depends on that closed form, which is why it can take exact gradient steps on the retrieved neighbours without an autodiff graph. Adding a third distribution here without also giving MbPAOutputNetwork.Gradient its residual would silently reuse one of these two.

Source: Sprechmann et al., “Memory-based Parameter Adaptation” (arXiv:1802.10542).

Fields

Field Summary
MbPAOutputDistribution Categorical Softmax over classes with a cross-entropy log-likelihood.
MbPAOutputDistribution Gaussian A unit-variance Gaussian, whose negative log-likelihood is one-half squared error up to a constant.
int value__