ParaformerObjective<T>
Models & Types · AiDotNet.LossFunctions
Paraformer’s training objective: gamma * L_CE + L_MAE (Gao et al., arXiv 2206.08317, Eq 6).
Inheritance: LossFunctionBase<T> → ParaformerObjective<T>
How It Works
Eq 6 states L_total = gamma*L_CE + L_MAE + L_Nwerr. This composes the first two terms; MWER
is a sampling-based sequence-level term requiring N-best decoding, so it is not folded in here.
The MAE term is what distinguishes this from plain cross-entropy: it supervises the CIF predictor’s predicted TOKEN COUNT against the target length. Paraformer §2.2/2.4 describe it as guiding “the predictor to convergence”, and without it that head trains unsupervised.
The predicted count is supplied by a callback rather than derived from
(predicted, target), because it is produced inside the CIF layer (as
sum_t alpha_t) and is not recoverable from the logits. The callback returns a TAPE-TRACKED
tensor, so gradient from this term flows back into the alpha predictor’s weights — which only became
possible once the CIF forward was expressed in differentiable Engine ops.
Constructors
| Constructor | Summary |
|---|---|
ParaformerObjective(LossFunctionBase<T> crossEntropy, Func<Tensor<T>> predictedTokenCount, Func<Tensor<T>> targetTokenCount, double ceWeight, double maeWeight) |
Creates the objective. |
Methods
| Method | Summary |
|---|---|
T CalculateLoss(Vector<T> predicted, Vector<T> actual) |
|
Tensor<T> ComputeTapeLoss(Tensor<T> predicted, Tensor<T> target) |