SNAILOptions<T, TInput, TOutput>
Options & Configuration · AiDotNet.MetaLearning.Options
Configuration options for SNAIL (Simple Neural Attentive Meta-Learner) algorithm.
Inheritance: ModelOptions → SNAILOptions<T, TInput, TOutput>
Implements: IMetaLearnerOptions<T>
For Beginners
SNAIL treats few-shot learning as a sequence problem:
The Idea: Feed all support examples (with their labels) one by one into the model, then feed query examples (without labels). The model learns to:
- Remember important examples (using temporal convolutions)
- Focus on relevant ones (using attention)
- Predict labels for query examples
Analogy: Imagine reading a detective story:
- Support examples are like clues presented throughout the story
- Temporal convolutions help you remember clues from different points in time
- Attention helps you focus on the most relevant clues when solving the mystery
- The query is the mystery to solve, using all the clues you’ve gathered
Architecture:
- Temporal Convolutions (TC): Capture short-range dependencies in the example sequence
- Causal Attention: Capture long-range dependencies by attending to any previous example
- Together: TC for local patterns + Attention for global patterns = powerful meta-learner
How It Works
SNAIL combines temporal convolutions with causal attention to perform sequence-to-sequence meta-learning. It processes support examples as a sequence and uses attention to selectively recall relevant examples when classifying query examples.
Reference: Mishra, N., Rohaninejad, M., Chen, X., & Abbeel, P. (2018). A Simple Neural Attentive Learner. ICLR 2018.
Constructors
| Constructor | Summary |
|---|---|
SNAILOptions(IFullModel<T, TInput, TOutput> metaModel) |
Initializes a new instance of SNAILOptions. |
Properties
| Property | Summary |
|---|---|
IEpisodicDataLoader<T, TInput, TOutput> DataLoader { get; set; } |
Gets or sets the episodic data loader. |
IFullModel<T, TInput, TOutput> MetaModel { get; set; } |
Gets or sets the base feature extractor model. |
IGradientBasedOptimizer<T, TInput, TOutput> InnerOptimizer { get; set; } |
Must stay null: this algorithm takes no plain gradient inner step (it adapts without one, or by its own update rule), so a supplied optimizer is rejected when the learner is built. |
IGradientBasedOptimizer<T, TInput, TOutput> MetaOptimizer { get; set; } |
Optimizer for the meta-update. |
ILossFunction<T> LossFunction { get; set; } |
Loss function for meta-training; null uses the default multi-class classification loss. |
bool EnableCheckpointing { get; set; } |
|
bool UseFirstOrder { get; set; } |
|
double DropoutRate { get; set; } |
Gets or sets the dropout rate for temporal convolutions and attention. |
double InnerLearningRate { get; set; } |
|
double OuterLearningRate { get; set; } |
|
double? GradientClipThreshold { get; set; } |
|
int AdaptationSteps { get; set; } |
Number of gradient steps taken on each task’s support set. |
int AttentionKeyDim { get; set; } |
Gets or sets the key dimension for attention. |
int AttentionValueDim { get; set; } |
Gets or sets the value dimension for attention. |
int CheckpointFrequency { get; set; } |
|
int EvaluationFrequency { get; set; } |
|
int EvaluationTasks { get; set; } |
|
int MaxSequenceLength { get; set; } |
Gets or sets the maximum sequence length the model can handle. |
int MetaBatchSize { get; set; } |
|
int NumAttentionHeads { get; set; } |
Gets or sets the number of attention heads for the causal attention blocks. |
int NumBlocks { get; set; } |
Gets or sets the number of TC+Attention blocks to stack. |
int NumMetaIterations { get; set; } |
|
int NumTCFilters { get; set; } |
Gets or sets the number of temporal convolution filters per block. |
int? RandomSeed { get; set; } |
Gets or sets the random seed. |
Methods
| Method | Summary |
|---|---|
IMetaLearnerOptions<T> Clone() |
|
bool IsValid() |