TransformerTeacherModel<T>

Models & Types · AiDotNet.KnowledgeDistillation.Teachers

Transformer-based teacher model that provides logits from transformer architectures.

Inheritance: TeacherModelBase<Vector<T>, Vector<T>, T> → TransformerTeacherModel<T>

How It Works

This wrapper takes a Func<Vector<T>, Vector<T>> forward-pass delegate and invokes it directly on every GetLogits call. The wrapper performs no caching or graph compilation itself — any optimizations (including Tensors’ AutoTracer auto-compile) depend on what the supplied delegate does internally.

For attention-based distillation strategies that need attention weights, implement a custom IDistillationStrategy that can extract attention from the underlying model.

Constructors

Constructor Summary
TransformerTeacherModel(Func<Vector<T>, Vector<T>> forwardFunc, int inputDimension, int outputDimension) Initializes a new instance of the TransformerTeacherModel class using a function delegate.

Properties

Property Summary
int OutputDimension { get; } Gets the output dimension.

Methods

Method Summary
Vector<T> GetLogits(Vector<T> input) Gets logits from the transformer model.