GraphAttentionNetwork<T>

Models & Types · AiDotNet.NeuralNetworks

Represents a Graph Attention Network (GAT) that uses attention mechanisms to process graph-structured data.

Inheritance: NeuralNetworkBase<T> → DeclaredModelLayoutBase<T> → GraphModelLayoutBase<T> → GraphAttentionNetwork<T>

For Beginners

GAT is like having a smart filter for your social network.

How it works:

  • Each node looks at its neighbors and decides which ones are most important
  • Important neighbors get more “attention” (higher weights)
  • Less relevant neighbors get less attention

Example - Movie Recommendations:

  • You’re a node connected to movies you’ve watched
  • Some movies better represent your taste than others
  • GAT learns to pay more attention to movies that define your preferences
  • Result: Better recommendations by focusing on what matters most

Key Features:

  • Multi-head attention: Multiple attention “perspectives” for robustness
  • Dynamic weights: Attention weights are learned, not fixed
  • Dropout support: Prevents overfitting during training
  • Configurable heads: Adjust number of attention heads for your task

Architecture: The standard GAT architecture consists of:

  1. Multiple GAT layers with attention mechanisms
  2. Optional dropout between layers
  3. Final classification or regression head

When to use GAT:

  • When some neighbors are more informative than others
  • When you need interpretable importance scores
  • For heterogeneous graphs where relationships vary in importance
  • Citation networks, social networks, knowledge graphs

How It Works

Graph Attention Networks introduce attention mechanisms to graph neural networks, allowing the model to learn which neighbors are most important for each node. Unlike GCN which treats all neighbors equally, GAT learns attention weights that determine how much each neighbor contributes to a node’s representation.

Example

var architecture = new NeuralNetworkArchitecture<float>(inputFeatures: 8, outputSize: 4);
var nodeFeatures = Tensor<float>.CreateRandom(new[] { 10, 16 });
var trainX = Tensor<float>.CreateRandom(4, 8);
var trainY = Tensor<float>.CreateRandom(4, 2);
var result = new AiModelBuilder<float, Tensor<float>, Tensor<float>>()
    .ConfigureModel(new GraphAttentionNetwork<float>(architecture))
    .Build(trainX, trainY);
var output = result.Predict(nodeFeatures);

Constructors

Constructor Summary
GraphAttentionNetwork() Initializes a new instance of the GraphAttentionNetwork class with specified architecture.
GraphAttentionNetwork(NeuralNetworkArchitecture<T> architecture, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer, ILossFunction<T> lossFunction, ILearningRateScheduler learningRateScheduler, GraphAttentionNetworkOptions options) Initializes the model from an already-resolved options instance.

Properties

Property Summary
bool IsLoRAEnabled { get; set; } Gets whether LoRA fine-tuning is currently enabled.
double DropoutRate { get; } Gets the dropout rate applied to attention coefficients during training.
int HiddenDim { get; } Gets the hidden dimension size for each layer.
int LoRARank { get; set; } Gets the LoRA rank when LoRA is enabled.
int NumHeads { get; } Gets the number of attention heads used in each GAT layer.
int NumLayers { get; } Gets the number of GAT layers in the network.

Methods

Method Summary
Dictionary<string, Tensor<T>> GetNamedLayerActivations(Tensor<T> input) Gets the intermediate activations from each layer, ensuring adjacency is set for graph layers.
List<Tensor<T>> GetAttentionWeights() Gets attention weights from all GAT layers for interpretability.
ModelMetadata<T> GetModelMetadata() Gets metadata about this model for serialization and identification.
Tensor<T> Forward(Tensor<T> nodeFeatures, Tensor<T> adjacencyMatrix) Performs a forward pass through the network with node features and adjacency matrix.
Tensor<T> ForwardForTraining(Tensor<T> input)
double Evaluate(Tensor<T> nodeFeatures, Tensor<T> adjacencyMatrix, Tensor<T> labels, bool[] testMask) Evaluates the model on test data and returns accuracy.
double GetLoRATrainablePercentage() Gets the percentage of parameters that are trainable when using LoRA.
int GetLoRAParameterCount() Gets the number of trainable LoRA parameters when LoRA is enabled.
long GetParameterCount() Gets the total number of trainable parameters in the network.
void DisableLoRA() Disables LoRA fine-tuning and restores original layers.
void EnableLoRAFineTuning(int rank, double alpha, bool freezeBaseLayers) Enables LoRA (Low-Rank Adaptation) fine-tuning for parameter-efficient training.
void MergeLoRAWeights() Merges LoRA weights into the base layers and disables LoRA mode.
void SetAdjacencyMatrix(Tensor<T> adjacencyMatrix) Sets the adjacency matrix for graph operations.
void TrainOnGraph(Tensor<T> nodeFeatures, Tensor<T> adjacencyMatrix, Tensor<T> labels, bool[] trainMask, int epochs, double learningRate) Trains the GAT network on graph-structured data.