FedKDLosses<T>
Models & Types · AiDotNet.FederatedLearning.Compression
The losses of one FedKD adaptive mutual distillation step, averaged over the batch.
Constructors
| Constructor | Summary |
|---|---|
FedKDLosses(T teacherTaskLoss, T studentTaskLoss, T teacherDistillationLoss, T studentDistillationLoss, T hiddenLoss) |
Creates the loss breakdown. |
Properties
| Property | Summary |
|---|---|
T HiddenLoss { get; } |
Adaptive hidden-state and attention loss shared by both models. |
T StudentDistillationLoss { get; } |
Adaptive distillation loss of the student, KL(y^t || y^s) / (L_t^t + L_t^s). |
T StudentTaskLoss { get; } |
Student (mentee) cross-entropy task loss. |
T StudentTotalLoss { get; } |
Total student loss L^s = L_t^s + L_d^s + L_h. |
T TeacherDistillationLoss { get; } |
Adaptive distillation loss of the teacher, KL(y^s || y^t) / (L_t^t + L_t^s). |
T TeacherTaskLoss { get; } |
Teacher (mentor) cross-entropy task loss. |
T TeacherTotalLoss { get; } |
Total teacher loss L^t = L_t^t + L_d^t + L_h. |