IntersampleAttentionLayer<T>
Layers · AiDotNet.NeuralNetworks.Layers
Intersample (Row) Attention for SAINT architecture.
Inheritance: LayerBase<T> → IntersampleAttentionLayer<T>
Implements: IShapeContract
For Beginners
While column attention looks at relationships between features within a single sample, intersample attention looks at relationships between different samples in the batch:
- Column attention: “How does age relate to income for this person?”
- Intersample attention: “How does this person compare to others in the batch?”
This allows the model to learn from the distribution of data, not just individual samples.
How It Works
Intersample attention allows samples in a batch to attend to each other, enabling the model to learn relationships between different data points. This is a key innovation in SAINT that helps with semi-supervised learning.
Constructors
| Constructor | Summary |
|---|---|
IntersampleAttentionLayer(int embeddingDim, int numHeads, double dropoutRate) |
Initializes intersample attention. |
Properties
| Property | Summary |
|---|---|
bool SupportsTraining { get; } |
Methods
| Method | Summary |
|---|---|
Dictionary<string, string> GetParameterRoles() |
Returns parameter roles for per-role optimizer configuration (e.g., weight decay exemption for biases). |
IReadOnlyList<ILayer<T>> GetSubLayers() |
|
IReadOnlyList<OutputAxisContract> OutputAxesFor(int inputRank) |
Derived from this type’s [TensorLayout] declarations. |
IReadOnlyList<Tensor<T>> GetTrainableParameters() |
Returns all trainable parameter tensors marked with [TrainableParameter]. |
void ResetState() |
Resets internal state. |
void SetTrainableParameters(IReadOnlyList<Tensor<T>> parameters) |
Replaces trainable parameter tensors (e.g., with ParameterBuffer views). |
void UpdateParameters(T learningRate) |
Updates parameters. |
void ZeroGrad() |
Clears all gradient fields discovered by convention ({paramName}Gradient). |