RelationNetworkAlgorithm<T, TInput, TOutput>
Models & Types · AiDotNet.MetaLearning.Algorithms
Implementation of Relation Networks algorithm for few-shot learning.
Inheritance: ModelBase<T, TInput, TOutput> → MetaLearnerBase<T, TInput, TOutput> → RelationNetworkAlgorithm<T, TInput, TOutput>
Implements: IGeneratedParameterRegistrar<T>
For Beginners
Relation Networks learn how to compare examples: a small network looks at a class’s examples and a new example side by side and says how related they are. The class with the highest relation score is the prediction.
How It Works
Relation Networks (Sung et al. 2018) learn the comparison itself. An embedding module f embeds the sample
(support) and query examples; for K-shot tasks each class’s embeddings are summed element-wise into one class
feature; a relation module g maps the concatenation C(f(x_i), f(x_j)) to a relation score
r_(i,j) in (0, 1) (eq. 1). Both modules are trained end to end to regress the scores onto the match
indicator with mean squared error, sum (r_(i,j) - 1(y_i == y_j))^2 (eq. 2).
What used to happen instead. The embedding module was never called: tensor inputs went to the relation module as raw features and matrix inputs became an empty tensor. The relation module was one dot product and a sigmoid. The loss was cross-entropy over a softmax of mean per-sample scores, and the embedding network’s update differentiated its own raw output against the labels, never the relation scores.
Extensions, all learned on the same objective and off by default except where the paper is the default:
the relation module’s architecture (RelationModuleType), how a class’s support examples meet a query
(RelationAggregationMethod), several relation heads averaged, a learned linear map of the
embeddings before pairing, and dropout inside the relation module while meta-training.
Constructors
| Constructor | Summary |
|---|---|
RelationNetworkAlgorithm(RelationNetworkOptions<T, TInput, TOutput> options) |
Initializes a new instance of the RelationNetworkAlgorithm class. |
Properties
| Property | Summary |
|---|---|
MetaLearningAlgorithmType AlgorithmType { get; } |
Methods
| Method | Summary |
|---|---|
IModel<TInput, TOutput, ModelMetadata<T>> Adapt(IMetaLearningTask<T, TInput, TOutput> task) |
|
T MetaTrain(TaskBatch<T, TInput, TOutput> taskBatch) |