SNAILOptions<T, TInput, TOutput>

Options & Configuration · AiDotNet.MetaLearning.Options

Configuration options for SNAIL (Simple Neural Attentive Meta-Learner) algorithm.

Inheritance: ModelOptions → SNAILOptions<T, TInput, TOutput>

Implements: IMetaLearnerOptions<T>

For Beginners

SNAIL treats few-shot learning as a sequence problem:

The Idea: Feed all support examples (with their labels) one by one into the model, then feed query examples (without labels). The model learns to:

  1. Remember important examples (using temporal convolutions)
  2. Focus on relevant ones (using attention)
  3. Predict labels for query examples

Analogy: Imagine reading a detective story:

  • Support examples are like clues presented throughout the story
  • Temporal convolutions help you remember clues from different points in time
  • Attention helps you focus on the most relevant clues when solving the mystery
  • The query is the mystery to solve, using all the clues you’ve gathered

Architecture:

  • Temporal Convolutions (TC): Capture short-range dependencies in the example sequence
  • Causal Attention: Capture long-range dependencies by attending to any previous example
  • Together: TC for local patterns + Attention for global patterns = powerful meta-learner

How It Works

SNAIL combines temporal convolutions with causal attention to perform sequence-to-sequence meta-learning. It processes support examples as a sequence and uses attention to selectively recall relevant examples when classifying query examples.

Reference: Mishra, N., Rohaninejad, M., Chen, X., & Abbeel, P. (2018). A Simple Neural Attentive Learner. ICLR 2018.

Constructors

Constructor Summary
SNAILOptions(IFullModel<T, TInput, TOutput> metaModel) Initializes a new instance of SNAILOptions.

Properties

Property Summary
IEpisodicDataLoader<T, TInput, TOutput> DataLoader { get; set; } Gets or sets the episodic data loader.
IFullModel<T, TInput, TOutput> MetaModel { get; set; } Gets or sets the base feature extractor model.
IGradientBasedOptimizer<T, TInput, TOutput> InnerOptimizer { get; set; } Must stay null: this algorithm takes no plain gradient inner step (it adapts without one, or by its own update rule), so a supplied optimizer is rejected when the learner is built.
IGradientBasedOptimizer<T, TInput, TOutput> MetaOptimizer { get; set; } Optimizer for the meta-update.
ILossFunction<T> LossFunction { get; set; } Loss function for meta-training; null uses the default multi-class classification loss.
bool EnableCheckpointing { get; set; }
bool UseFirstOrder { get; set; }
double DropoutRate { get; set; } Gets or sets the dropout rate for temporal convolutions and attention.
double InnerLearningRate { get; set; }
double OuterLearningRate { get; set; }
double? GradientClipThreshold { get; set; }
int AdaptationSteps { get; set; } Number of gradient steps taken on each task’s support set.
int AttentionKeyDim { get; set; } Gets or sets the key dimension for attention.
int AttentionValueDim { get; set; } Gets or sets the value dimension for attention.
int CheckpointFrequency { get; set; }
int EvaluationFrequency { get; set; }
int EvaluationTasks { get; set; }
int MaxSequenceLength { get; set; } Gets or sets the maximum sequence length the model can handle.
int MetaBatchSize { get; set; }
int NumAttentionHeads { get; set; } Gets or sets the number of attention heads for the causal attention blocks.
int NumBlocks { get; set; } Gets or sets the number of TC+Attention blocks to stack.
int NumMetaIterations { get; set; }
int NumTCFilters { get; set; } Gets or sets the number of temporal convolution filters per block.
int? RandomSeed { get; set; } Gets or sets the random seed.

Methods

Method Summary
IMetaLearnerOptions<T> Clone()
bool IsValid()