GraphTransformerLayer<T>
Layers · AiDotNet.NeuralNetworks.Layers
Implements Graph Transformer layer using self-attention mechanisms on graph-structured data.
Inheritance: LayerBase<T> → GraphTransformerLayer<T>
Implements: IGraphConvolutionLayer<T>, IShapeContract
For Beginners
Graph Transformers combine the power of transformers with graph structure.
Think of it like a meeting where:
- Standard transformers: Everyone can talk to everyone equally
- Graph transformers: People connected in the organizational chart get priority
Key advantages:
- Captures long-range dependencies in graphs
- More flexible than fixed neighborhood aggregation
- Can attend to any node, not just immediate neighbors
- Learns importance of connections dynamically
Use cases:
- Large molecules: Atoms far apart but chemically important
- Social networks: Identifying influential users across communities
- Knowledge graphs: Multi-hop reasoning
- Program analysis: Understanding code dependencies
Example: In a citation network, a paper can learn from:
- Direct citations (immediate neighbors)
- Indirectly related papers (through attention)
- Important papers even if not directly cited
How It Works
Graph Transformers apply the transformer architecture to graphs by treating graph structure as a bias in the attention mechanism. Unlike standard transformers that process sequences, Graph Transformers incorporate graph connectivity through:
- Structural encodings (e.g., Laplacian eigenvectors)
- Attention biasing based on graph structure
- Relative positional encodings for graph nodes
The attention computation is: Attention(Q, K, V) = softmax((QK^T + B)/√d_k)V where B is a learned bias based on graph structure.
Constructors
| Constructor | Summary |
|---|---|
GraphTransformerLayer(int inputFeatures, int outputFeatures, int numHeads, int headDim, bool useStructuralEncoding, double dropoutRate, IActivationFunction<T> activationFunction, IActivationFunction<T> ffnActivation) |
Initializes a new instance of the GraphTransformerLayer class with a vector activation function. |
GraphTransformerLayer(int inputFeatures, int outputFeatures, int numHeads, int headDim, bool useStructuralEncoding, double dropoutRate, IVectorActivationFunction<T> vectorActivationFunction, IActivationFunction<T> ffnActivation) |
Initializes a new instance of the GraphTransformerLayer class with a vector activation function. |
Properties
| Property | Summary |
|---|---|
bool SupportsTraining { get; } |
|
int InputFeatures { get; } |
|
int OutputFeatures { get; } |
Methods
| Method | Summary |
|---|---|
Dictionary<string, string> GetParameterRoles() |
Returns parameter roles for per-role optimizer configuration (e.g., weight decay exemption for biases). |
IReadOnlyList<OutputAxisContract> OutputAxesFor(int inputRank) |
|
IReadOnlyList<Tensor<T>> GetTrainableParameters() |
Returns all trainable parameter tensors marked with [TrainableParameter]. |
IReadOnlyList<ValueTuple<string, Tensor<T>>> GetRegisteredBuffers() |
|
Tensor<T> GetAdjacencyMatrix() |
|
void ClearGradients() |
|
void ResetState() |
|
void SetAdjacencyMatrix(Tensor<T> adjacencyMatrix) |
|
void SetTrainableParameters(IReadOnlyList<Tensor<T>> parameters) |
Replaces trainable parameter tensors (e.g., with ParameterBuffer views). |
void UpdateParameters(T learningRate) |
|
void ZeroGrad() |
Clears all gradient fields discovered by convention ({paramName}Gradient). |