FEATOptions<T, TInput, TOutput>
Options & Configuration · AiDotNet.MetaLearning.Options
Configuration options for FEAT (Few-shot Embedding Adaptation with Transformer) (Ye et al., CVPR 2020).
Inheritance: ModelOptions → FEATOptions<T, TInput, TOutput>
Implements: IMetaLearnerOptions<T>
For Beginners
FEAT improves upon ProtoNets by making prototypes “task-aware”:
Problem with ProtoNets: Each class prototype is computed independently (mean of support features). But knowing about OTHER classes in the task could help: if class A and B are similar, their prototypes should be pushed apart.
FEAT’s solution: Use a transformer to let prototypes “talk to each other”:
- Compute initial prototypes (like ProtoNets)
- Feed all prototypes through a set-to-set transformer
- The transformer adjusts each prototype based on the others
- Result: task-adapted prototypes that are more discriminative
Analogy: Imagine placing labels on a map:
- ProtoNets: Places each label independently
- FEAT: Adjusts labels so they’re evenly spread and don’t overlap
How It Works
FEAT adapts embeddings to be task-specific using a set-to-set transformer that takes class prototypes as input and outputs task-adapted prototypes. The transformer captures inter-class relationships to produce more discriminative prototypes.
Reference: Ye, H.J., Hu, H., Zhan, D.C., & Sha, F. (2020). Few-Shot Learning via Embedding Adaptation with Set-to-Set Functions. CVPR 2020.
Constructors
| Constructor | Summary |
|---|---|
FEATOptions(IFullModel<T, TInput, TOutput> metaModel) |
Initializes a new instance of FEATOptions. |
Properties
| Property | Summary |
|---|---|
IEpisodicDataLoader<T, TInput, TOutput> DataLoader { get; set; } |
Gets or sets the episodic data loader. |
IFullModel<T, TInput, TOutput> MetaModel { get; set; } |
Gets or sets the feature extractor model. |
IGradientBasedOptimizer<T, TInput, TOutput> InnerOptimizer { get; set; } |
Must stay null: this algorithm takes no plain gradient inner step (it adapts without one, or by its own update rule), so a supplied optimizer is rejected when the learner is built. |
IGradientBasedOptimizer<T, TInput, TOutput> MetaOptimizer { get; set; } |
Optimizer for the meta-update. |
ILossFunction<T> LossFunction { get; set; } |
Loss function for meta-training; null uses the default multi-class classification loss. |
bool EnableCheckpointing { get; set; } |
|
bool UseFirstOrder { get; set; } |
|
double ContrastiveWeight { get; set; } |
Gets or sets the balance weight between contrastive loss and classification loss. |
double InnerLearningRate { get; set; } |
|
double OuterLearningRate { get; set; } |
|
double Temperature { get; set; } |
Gets or sets the temperature for similarity computation. |
double? GradientClipThreshold { get; set; } |
|
int AdaptationSteps { get; set; } |
Number of gradient steps taken on each task’s support set. |
int CheckpointFrequency { get; set; } |
|
int EvaluationFrequency { get; set; } |
|
int EvaluationTasks { get; set; } |
|
int MetaBatchSize { get; set; } |
|
int NumMetaIterations { get; set; } |
|
int NumTransformerHeads { get; set; } |
Gets or sets the number of attention heads in the set-to-set transformer. |
int NumTransformerLayers { get; set; } |
Gets or sets the number of transformer layers. |
int TransformerDim { get; set; } |
Gets or sets the feature dimension for the set-to-set transformer. |
int? RandomSeed { get; set; } |
Gets or sets the random seed. |
Methods
| Method | Summary |
|---|---|
IMetaLearnerOptions<T> Clone() |
|
bool IsValid() |