TransformerTeacherModel<T>
Models & Types · AiDotNet.KnowledgeDistillation.Teachers
Transformer-based teacher model that provides logits from transformer architectures.
Inheritance: TeacherModelBase<Vector<T>, Vector<T>, T> → TransformerTeacherModel<T>
How It Works
This wrapper takes a Func<Vector<T>, Vector<T>> forward-pass
delegate and invokes it directly on every GetLogits call.
The wrapper performs no caching or graph compilation itself — any
optimizations (including Tensors’ AutoTracer auto-compile) depend on what
the supplied delegate does internally.
For attention-based distillation strategies that need attention weights, implement a custom IDistillationStrategy that can extract attention from the underlying model.
Constructors
| Constructor | Summary |
|---|---|
TransformerTeacherModel(Func<Vector<T>, Vector<T>> forwardFunc, int inputDimension, int outputDimension) |
Initializes a new instance of the TransformerTeacherModel class using a function delegate. |
Properties
| Property | Summary |
|---|---|
int OutputDimension { get; } |
Gets the output dimension. |
Methods
| Method | Summary |
|---|---|
Vector<T> GetLogits(Vector<T> input) |
Gets logits from the transformer model. |