ParaformerObjective<T>

Models & Types · AiDotNet.LossFunctions

Paraformer’s training objective: gamma * L_CE + L_MAE (Gao et al., arXiv 2206.08317, Eq 6).

Inheritance: LossFunctionBase<T> → ParaformerObjective<T>

How It Works

Eq 6 states L_total = gamma*L_CE + L_MAE + L_Nwerr. This composes the first two terms; MWER is a sampling-based sequence-level term requiring N-best decoding, so it is not folded in here.

The MAE term is what distinguishes this from plain cross-entropy: it supervises the CIF predictor’s predicted TOKEN COUNT against the target length. Paraformer §2.2/2.4 describe it as guiding “the predictor to convergence”, and without it that head trains unsupervised.

The predicted count is supplied by a callback rather than derived from (predicted, target), because it is produced inside the CIF layer (as sum_t alpha_t) and is not recoverable from the logits. The callback returns a TAPE-TRACKED tensor, so gradient from this term flows back into the alpha predictor’s weights — which only became possible once the CIF forward was expressed in differentiable Engine ops.

Constructors

Constructor Summary
ParaformerObjective(LossFunctionBase<T> crossEntropy, Func<Tensor<T>> predictedTokenCount, Func<Tensor<T>> targetTokenCount, double ceWeight, double maeWeight) Creates the objective.

Methods

Method Summary
T CalculateLoss(Vector<T> predicted, Vector<T> actual)
Tensor<T> ComputeTapeLoss(Tensor<T> predicted, Tensor<T> target)