RelationNetworkAlgorithm<T, TInput, TOutput>

Models & Types · AiDotNet.MetaLearning.Algorithms

Implementation of Relation Networks algorithm for few-shot learning.

Inheritance: ModelBase<T, TInput, TOutput> → MetaLearnerBase<T, TInput, TOutput> → RelationNetworkAlgorithm<T, TInput, TOutput>

Implements: IGeneratedParameterRegistrar<T>

For Beginners

Relation Networks learn how to compare examples: a small network looks at a class’s examples and a new example side by side and says how related they are. The class with the highest relation score is the prediction.

How It Works

Relation Networks (Sung et al. 2018) learn the comparison itself. An embedding module f embeds the sample (support) and query examples; for K-shot tasks each class’s embeddings are summed element-wise into one class feature; a relation module g maps the concatenation C(f(x_i), f(x_j)) to a relation score r_(i,j) in (0, 1) (eq. 1). Both modules are trained end to end to regress the scores onto the match indicator with mean squared error, sum (r_(i,j) - 1(y_i == y_j))^2 (eq. 2).

What used to happen instead. The embedding module was never called: tensor inputs went to the relation module as raw features and matrix inputs became an empty tensor. The relation module was one dot product and a sigmoid. The loss was cross-entropy over a softmax of mean per-sample scores, and the embedding network’s update differentiated its own raw output against the labels, never the relation scores.

Extensions, all learned on the same objective and off by default except where the paper is the default: the relation module’s architecture (RelationModuleType), how a class’s support examples meet a query (RelationAggregationMethod), several relation heads averaged, a learned linear map of the embeddings before pairing, and dropout inside the relation module while meta-training.

Constructors

Constructor Summary
RelationNetworkAlgorithm(RelationNetworkOptions<T, TInput, TOutput> options) Initializes a new instance of the RelationNetworkAlgorithm class.

Properties

Property Summary
MetaLearningAlgorithmType AlgorithmType { get; }

Methods

Method Summary
IModel<TInput, TOutput, ModelMetadata<T>> Adapt(IMetaLearningTask<T, TInput, TOutput> task)
T MetaTrain(TaskBatch<T, TInput, TOutput> taskBatch)