GAT (Graph Attention Network) adds: learned attention weights between neighbors

How do computers learn what to focus on in a complex network?

Imagine you're trying to find the best route through a crowded city. You can't see every street, but you want to know which paths are less crowded.

Picture each street as a node in a network, with the crowd level as the weight of the connection. The idea is to let the computer figure out which streets (connections) are more important to avoid crowds.

Example

If Street A connects to Street B and Street C, but Street B is always less crowded, the computer learns to pay more attention to Street B when choosing a route.

Remember this

The computer learns to focus on the most relevant connections, like paying attention to less crowded streets in a city.

Text adapted from Wikipedia, licensed under CC BY-SA 4.0.

Related concepts

message passing does in GNNs: each node aggregates features from its neighbors

Can you imagine a social network where your connections influence your opinions?

Graduate Aptitude Test in Engineering

GATE exam assesses engineering and science undergraduate subjects for postgraduate admissions in India

GCN (Graph Convolutional Network) does: spectral convolution approximated by neighbor averaging

How can we predict a molecule's medical efficacy?

grouped query attention (GQA) does

GQA shares KV heads across multiple Q heads for efficient parameter usage

2022 in science

Why do Transformers sometimes seem to 'ignore' irrelevant parts of the input?

Machine learning in bioinformatics

How do Transformers understand what's important in a sentence?

Swipe through 100 ML concepts daily

Open Pocket Polymath