GraphTransformerLayer<T>

Layers · AiDotNet.NeuralNetworks.Layers

Implements Graph Transformer layer using self-attention mechanisms on graph-structured data.

Inheritance: LayerBase<T> → GraphTransformerLayer<T>

Implements: IGraphConvolutionLayer<T>, IShapeContract

For Beginners

Graph Transformers combine the power of transformers with graph structure.

Think of it like a meeting where:

  • Standard transformers: Everyone can talk to everyone equally
  • Graph transformers: People connected in the organizational chart get priority

Key advantages:

  • Captures long-range dependencies in graphs
  • More flexible than fixed neighborhood aggregation
  • Can attend to any node, not just immediate neighbors
  • Learns importance of connections dynamically

Use cases:

  • Large molecules: Atoms far apart but chemically important
  • Social networks: Identifying influential users across communities
  • Knowledge graphs: Multi-hop reasoning
  • Program analysis: Understanding code dependencies

Example: In a citation network, a paper can learn from:

  • Direct citations (immediate neighbors)
  • Indirectly related papers (through attention)
  • Important papers even if not directly cited

How It Works

Graph Transformers apply the transformer architecture to graphs by treating graph structure as a bias in the attention mechanism. Unlike standard transformers that process sequences, Graph Transformers incorporate graph connectivity through:

  1. Structural encodings (e.g., Laplacian eigenvectors)
  2. Attention biasing based on graph structure
  3. Relative positional encodings for graph nodes

The attention computation is: Attention(Q, K, V) = softmax((QK^T + B)/√d_k)V where B is a learned bias based on graph structure.

Constructors

Constructor Summary
GraphTransformerLayer(int inputFeatures, int outputFeatures, int numHeads, int headDim, bool useStructuralEncoding, double dropoutRate, IActivationFunction<T> activationFunction, IActivationFunction<T> ffnActivation) Initializes a new instance of the GraphTransformerLayer class with a vector activation function.
GraphTransformerLayer(int inputFeatures, int outputFeatures, int numHeads, int headDim, bool useStructuralEncoding, double dropoutRate, IVectorActivationFunction<T> vectorActivationFunction, IActivationFunction<T> ffnActivation) Initializes a new instance of the GraphTransformerLayer class with a vector activation function.

Properties

Property Summary
bool SupportsTraining { get; }
int InputFeatures { get; }
int OutputFeatures { get; }

Methods

Method Summary
Dictionary<string, string> GetParameterRoles() Returns parameter roles for per-role optimizer configuration (e.g., weight decay exemption for biases).
IReadOnlyList<OutputAxisContract> OutputAxesFor(int inputRank)
IReadOnlyList<Tensor<T>> GetTrainableParameters() Returns all trainable parameter tensors marked with [TrainableParameter].
IReadOnlyList<ValueTuple<string, Tensor<T>>> GetRegisteredBuffers()
Tensor<T> GetAdjacencyMatrix()
void ClearGradients()
void ResetState()
void SetAdjacencyMatrix(Tensor<T> adjacencyMatrix)
void SetTrainableParameters(IReadOnlyList<Tensor<T>> parameters) Replaces trainable parameter tensors (e.g., with ParameterBuffer views).
void UpdateParameters(T learningRate)
void ZeroGrad() Clears all gradient fields discovered by convention ({paramName}Gradient).