CAVIAAlgorithm<T, TInput, TOutput>
Models & Types · AiDotNet.MetaLearning.Algorithms
Implementation of CAVIA (Fast Context Adaptation via Meta-Learning) for few-shot learning.
Inheritance: ModelBase<T, TInput, TOutput> → MetaLearnerBase<T, TInput, TOutput> → CAVIAAlgorithm<T, TInput, TOutput>
For Beginners
CAVIA is a smarter version of MAML that adapts faster:
How it works:
- The model has two kinds of parameters:
- Body parameters (shared across all tasks) - these are the model’s “core skills”
- Context parameters (adapted per task) - these are task-specific adjustments
- For a new task, only context parameters are updated (much faster!)
- The body parameters improve over time across all tasks
- Context is a small vector concatenated with/added to the input
Simple example:
- Body params: How to recognize shapes, edges, textures (shared knowledge)
- Context params: “Right now I’m looking at animals” vs “Right now I’m looking at vehicles”
- For each new task, only adjust what kind of thing you’re looking at
- Your fundamental perception skills stay the same
Why it’s better than MAML:
- MAML adapts ALL parameters (expensive, O(P) where P = total params)
- CAVIA adapts only context (cheap, O(C) where C = context_dim, C << P)
- Less prone to meta-overfitting (fewer adapted parameters)
- Mathematically cleaner separation of shared vs. task-specific knowledge
How It Works
CAVIA separates model parameters into shared body parameters and task-specific context parameters. Only the small context vector is adapted during the inner loop, making CAVIA significantly faster than full MAML while achieving comparable performance.
Algorithm - CAVIA:
Key Insights:
- Separation of Concerns: Body parameters capture shared structure across tasks,
while context parameters capture task-specific information.
- Efficient Adaptation: Only adapting the small context vector means the inner
loop is O(context_dim) instead of O(total_params), typically 100x-1000x cheaper.
- Reduced Meta-Overfitting: Fewer adapted parameters means less risk of overfitting
the meta-learner to the training tasks.
- Interpretable Context: The context vector provides a low-dimensional representation
of what makes each task unique.
Reference: Zintgraf, L. M., Shiarli, K., Kurin, V., Hofmann, K., & Whiteson, S. (2019). Fast Context Adaptation via Meta-Learning. ICML 2019.
Constructors
| Constructor | Summary |
|---|---|
CAVIAAlgorithm(CAVIAOptions<T, TInput, TOutput> options) |
Initializes a new instance of the CAVIAAlgorithm class. |
Properties
| Property | Summary |
|---|---|
MetaLearningAlgorithmType AlgorithmType { get; } |
Gets the algorithm type identifier for this meta-learner. |
Methods
| Method | Summary |
|---|---|
IModel<TInput, TOutput, ModelMetadata<T>> Adapt(IMetaLearningTask<T, TInput, TOutput> task) |
Adapts to a new task by learning task-specific context parameters. |
ModelOptions GetOptions() |
|
T MetaTrain(TaskBatch<T, TInput, TOutput> taskBatch) |
Performs one meta-training step using CAVIA’s context adaptation strategy. |