AttentionMatchingMode

Enums · AiDotNet.KnowledgeDistillation.Strategies

Defines how to match attention patterns between teacher and student.

Fields

Field Summary
AttentionMatchingMode Cosine Cosine similarity - focuses on direction/pattern rather than magnitude.
AttentionMatchingMode KL KL Divergence - treats attention as probability distribution, preserves structure.
AttentionMatchingMode MSE Mean Squared Error - simple, fast, treats all attention weights equally.
int value__