GraphAttentionLayer<T>
Layers · AiDotNet.NeuralNetworks.Layers
Implements Graph Attention Network (GAT) layer for processing graph-structured data with attention mechanisms.
Inheritance: LayerBase<T> → GraphAttentionLayer<T>
Implements: IGraphConvolutionLayer<T>, IShapeContract
How It Works
Graph Attention Networks (GAT) introduced by Veličković et al. use attention mechanisms to learn the relative importance of neighboring nodes. Unlike standard GCN which treats all neighbors equally, GAT can assign different weights to different neighbors, allowing the model to focus on the most relevant connections. The layer uses multi-head attention for robustness and expressiveness.
The attention mechanism computes: α_ij = softmax(LeakyReLU(a^T [Wh_i || Wh_j])) where α_ij is the attention coefficient from node j to node i, W is a weight matrix, h_i and h_j are node features, a is the attention vector, and || denotes concatenation.
Production-Ready Features:
- Fully vectorized operations using IEngine for GPU acceleration
- Tensor-based weights for all parameters
- Dual backward pass: BackwardManual() for efficiency, BackwardViaAutodiff() for accuracy
- Full gradient computation through attention mechanism
- Complete GetParameters()/SetParameters() for model persistence
Constructors
| Constructor | Summary |
|---|---|
GraphAttentionLayer(int inputFeatures, int outputFeatures, int numHeads, double alpha, double dropoutRate, IActivationFunction<T> activationFunction, IInitializationStrategy<T> initializationStrategy, bool concatenateHeads) |
Initializes a new instance of the GraphAttentionLayer class. |
Properties
| Property | Summary |
|---|---|
bool ConcatenateHeads { get; } |
Gets whether the per-head outputs are concatenated (hidden-layer GAT rule) instead of averaged (prediction-layer GAT rule). |
bool SupportsTraining { get; } |
|
bool UsesSparseAggregation { get; } |
Gets whether sparse (edge-based) aggregation is currently enabled. |
double Alpha { get; } |
Gets the LeakyReLU negative-slope used inside the attention score computation. |
double DropoutRate { get; } |
Gets the dropout rate applied to attention coefficients during training. |
int InputFeatures { get; } |
|
int NumHeads { get; } |
Gets the number of attention heads used in multi-head attention. |
int OutputFeatures { get; } |
Methods
| Method | Summary |
|---|---|
Dictionary<string, string> GetParameterRoles() |
Returns parameter roles for per-role optimizer configuration (e.g., weight decay exemption for biases). |
IReadOnlyList<OutputAxisContract> OutputAxesFor(int inputRank) |
|
IReadOnlyList<Tensor<T>> GetTrainableParameters() |
Returns all trainable parameter tensors marked with [TrainableParameter]. |
IReadOnlyList<ValueTuple<string, Tensor<T>>> GetRegisteredBuffers() |
|
Tensor<T> GetAdjacencyMatrix() |
|
void ClearEdges() |
Clears the edge list and switches back to dense adjacency matrix aggregation. |
void ClearGradients() |
|
void ResetState() |
|
void SetAdjacencyMatrix(Tensor<T> adjacencyMatrix) |
|
void SetEdges(Tensor<int> sourceIndices, Tensor<int> targetIndices) |
Sets the edge list representation of the graph structure for sparse aggregation. |
void SetTrainableParameters(IReadOnlyList<Tensor<T>> parameters) |
Replaces trainable parameter tensors (e.g., with ParameterBuffer views). |
void UpdateParameters(T learningRate) |
|
void ZeroGrad() |
Clears all gradient fields discovered by convention ({paramName}Gradient). |