IntersampleAttentionLayer<T>

Layers · AiDotNet.NeuralNetworks.Layers

Intersample (Row) Attention for SAINT architecture.

Inheritance: LayerBase<T> → IntersampleAttentionLayer<T>

Implements: IShapeContract

For Beginners

While column attention looks at relationships between features within a single sample, intersample attention looks at relationships between different samples in the batch:

  • Column attention: “How does age relate to income for this person?”
  • Intersample attention: “How does this person compare to others in the batch?”

This allows the model to learn from the distribution of data, not just individual samples.

How It Works

Intersample attention allows samples in a batch to attend to each other, enabling the model to learn relationships between different data points. This is a key innovation in SAINT that helps with semi-supervised learning.

Constructors

Constructor Summary
IntersampleAttentionLayer(int embeddingDim, int numHeads, double dropoutRate) Initializes intersample attention.

Properties

Property Summary
bool SupportsTraining { get; }

Methods

Method Summary
Dictionary<string, string> GetParameterRoles() Returns parameter roles for per-role optimizer configuration (e.g., weight decay exemption for biases).
IReadOnlyList<ILayer<T>> GetSubLayers()
IReadOnlyList<OutputAxisContract> OutputAxesFor(int inputRank) Derived from this type’s [TensorLayout] declarations.
IReadOnlyList<Tensor<T>> GetTrainableParameters() Returns all trainable parameter tensors marked with [TrainableParameter].
void ResetState() Resets internal state.
void SetTrainableParameters(IReadOnlyList<Tensor<T>> parameters) Replaces trainable parameter tensors (e.g., with ParameterBuffer views).
void UpdateParameters(T learningRate) Updates parameters.
void ZeroGrad() Clears all gradient fields discovered by convention ({paramName}Gradient).