GCN, GraphSAGE, GAT and Temporal GNNs

A graph neural network has exactly one job that matters: take a node, look at its neighbours, and squeeze them into a better description of the node. A bank is partly defined by who it lends to; a wallet is partly defined by who it transacts with; a supplier is partly defined by who it ships to. The whole field is just a long argument about how to do that squeezing.

That argument produced a lineage. GCN averages neighbours with a fixed recipe. GraphSAGE samples a handful of them and learns the recipe, crucially without needing the whole graph up front. GAT lets the node decide which neighbours deserve more of its attention. Temporal GNNs admit the embarrassing truth that in finance the graph itself is a moving target — yesterday’s lending network is not today’s.

This lesson walks that lineage in order. Each architecture is a different answer to the same question, and the differences are exactly the things that decide whether your model survives contact with a real, growing, non-stationary financial network.

Why there’s a lineage at all

Analogy. Picture a node as a person deciding what to think after talking to their contacts. GCN is the person who gives everyone an equal, pre-set say (with a polite adjustment for how popular each contact is). GraphSAGE is the person who can’t possibly talk to all their contacts, so they poll a random sample and learn how to weigh the gossip. GAT is the person who listens hardest to the contacts who actually matter and tunes out the noise. Same room, three temperaments.

Definition. A GNN layer updates every node’s hidden vector $h_v^{(k)}$ from the previous layer’s vectors using two operations: an aggregate over the neighbourhood $\mathcal{N}(v)$, and a combine with the node’s own previous vector $h_v^{(k-1)}$. Stacking $K$ such layers lets information flow $K$ hops: after layer $K$, node $v$‘s embedding has been touched by everything within $K$ steps in the graph. Generically,

$h_v^{(k)} = \text{COMBINE}\Big( h_v^{(k-1)},\ \text{AGG}\big(\{\,h_u^{(k-1)} : u \in \mathcal{N}(v)\,\}\big) \Big).$

Every architecture in this lesson is a specific choice of AGG and COMBINE — and a specific stance on whether the graph is fixed, growing, weighted, or moving. Here is the whole lineage on one axis before we dig in:

GCN — fixed degree-normalised averaging

Analogy. GCN is a town-hall meeting with a strict, pre-printed rule for who speaks and how loudly: everyone gets a turn, but the rule quietly turns down the volume of the loudmouths who know everyone (high-degree nodes) so they don’t dominate. Nobody chose the rule for tonight’s meeting — it was the same rule last week and will be next week.

Definition. The Graph Convolutional Network layer applies one shared linear transform $W$ to every node, then aggregates over the neighbourhood including the node itself (the self-loop), each term weighted by a fixed symmetric degree normalisation, then passes the sum through a nonlinearity $\sigma$:

$h_v^{(k)} = \sigma\!\left( \sum_{u \in \mathcal{N}(v) \cup \{v\}} \frac{1}{\sqrt{\deg(v)\,\deg(u)}}\, W\, h_u^{(k-1)} \right).$

The factor $1/\sqrt{\deg(v)\,\deg(u)}$ is the heart of it. It is not learned — it is read straight off the graph’s structure. It down-weights edges touching high-degree nodes so that a hub’s flood of connections doesn’t drown out a leaf node’s single meaningful link. (For the self-loop term $u = v$ the degree includes the added self-edge, but we keep the arithmetic simple below.)

Worked example. Let node $v$ be a small bank with two neighbours, $a$ and $b$, and a self-loop. Suppose (after adding self-loops) the degrees are $\deg(v) = 3$, $\deg(a) = 3$, $\deg(b) = 12$. Take a one-dimensional feature so the arithmetic is visible, with $W = 1$ and $\sigma$ the identity, and previous-layer values $h_v = 10$, $h_a = 4$, $h_b = 1$. The three edge weights are:

Sum: $3.333 + 1.333 + 0.167 = 4.833$. So $h_v^{(k)} \approx 4.83$. Notice that the high-degree neighbour $b$ got the smallest weight (0.167 vs 0.333) purely because it is a hub — the degree normalisation did that automatically, before any learning. The bank’s own state (3.333) still dominates, but its two neighbours pulled the new value down toward theirs.

When to use it

Reach for GCN when the graph is fixed and fully known at training time, reasonably homogeneous, and you want a strong, cheap, well-understood baseline — node classification on a static interbank network snapshot, say, where every institution is already in the graph. It is the linear-regression-equivalent of GNNs: the thing you should beat before you complicate your life. If nodes will keep arriving, GCN’s transductive nature is a sign to move down the lineage.

GraphSAGE — sample, aggregate, generalise

Analogy. A new analyst joins a firm with 50,000 client relationships. They cannot read every file, so they learn a reusable habit: “sample a dozen relationships, summarise them, staple that summary to what I already know about the client.” That habit works on client #50,001 — whom they’ve never met — because they learned the procedure, not a memorised answer for each existing client. That is GraphSAGE: SAmple and aggreGatE.

Definition. A GraphSAGE layer (1) samples a fixed-size set of neighbours from $\mathcal{N}(v)$, (2) aggregates their previous vectors with a learnable aggregator (mean, pooling, or even an LSTM), and (3) concatenates the result with the node’s own previous vector before projecting through $W$ and a nonlinearity:

$h_v^{(k)} = \sigma\!\Big( W \cdot \text{CONCAT}\big(\, h_v^{(k-1)},\ \text{AGG}(\{\,h_u^{(k-1)} : u \in \mathcal{S}(v)\,\})\,\big) \Big),$

where $\mathcal{S}(v)$ is the sampled subset of neighbours. The two design moves that matter: concatenate (self and neighbours are kept as distinct halves, not blended into one average as in GCN), and sample (a fixed fan-out keeps cost bounded even for a hub with a million edges). Because the layer learns a function rather than a per-node embedding, the same trained weights embed nodes that did not exist at training time. This is the property called inductive learning.

Worked example (mean aggregator). Node $v$ has feature $h_v = 6$ and four neighbours with values $\{2, 8, 5, 9\}$. We sample two of them — say we draw $\{8, 5\}$. The mean aggregate is $(8 + 5)/2 = 6.5$. Concatenate with self: the layer input is the pair $(6,\ 6.5)$. With a tiny projection $W = [0.5,\ 0.5]$ (so it averages the two halves) and identity $\sigma$:

$h_v^{(k)} = 0.5 \cdot 6 + 0.5 \cdot 6.5 = 3.0 + 3.25 = 6.25.$

Now suppose the next training step samples a different pair, $\{2, 9\}$: the aggregate is $(2+9)/2 = 5.5$, and $h_v^{(k)} = 0.5 \cdot 6 + 0.5 \cdot 5.5 = 5.75$. Same node, same weights, different answer — because the sample changed.

When to use it

GraphSAGE is the realistic finance default, because financial graphs grow: new banks charter, new wallets appear, new suppliers enter the network every day, and you need to score them without retraining the world. Use it when the node set is open-ended, when hubs are too large to aggregate fully, or when you train on one time period and must deploy on later, larger graphs. If your graph is genuinely fixed and small, GCN’s exactness may beat SAGE’s sampling noise — but that is the exception in finance, not the rule.

GAT — learn which neighbours matter

Analogy. GCN gives everyone a degree-adjusted but otherwise equal vote. GAT hires a staff that reads the room and decides, per conversation, whose opinion to weight: the counterparty exposing you to $2 billion gets leaned on; the $10,000 nuisance account gets a polite nod. Crucially, the staff learned how to allocate that attention from past data — it wasn’t told “size equals importance,” it figured out what importance looks like.

Definition. A Graph Attention Network layer computes, for each edge $v \leftarrow u$, a learned, normalised attention coefficient $\alpha_{vu}$, then aggregates neighbours as an attention-weighted sum. The coefficients come from a softmax over learned scores $e_{vu}$:

$\alpha_{vu} = \frac{\exp(e_{vu})}{\sum_{w \in \mathcal{N}(v) \cup \{v\}} \exp(e_{vw})}, \qquad h_v^{(k)} = \sigma\!\left( \sum_{u \in \mathcal{N}(v) \cup \{v\}} \alpha_{vu}\, W\, h_u^{(k-1)} \right).$

The raw score $e_{vu}$ is produced by a small learnable function of the two transformed endpoint vectors $W h_v$ and $W h_u$ (in the original GAT, a single-layer feedforward on their concatenation, with a LeakyReLU). The softmax guarantees the weights over a node’s neighbourhood are non-negative and sum to one. Multi-head attention runs several independent attention mechanisms in parallel and concatenates (or averages) their outputs, which stabilises training the way multiple analysts cross-check each other.

Worked example (softmax over 3 neighbours). Node $v$ has three neighbours with raw learned scores $e = \{2.0,\ 1.0,\ 0.0\}$ and per-neighbour values $h = \{10,\ 4,\ 1\}$. Exponentiate: $e^{2.0} \approx 7.389$, $e^{1.0} \approx 2.718$, $e^{0.0} = 1.000$. Their sum is $7.389 + 2.718 + 1.000 = 11.107$. The attention weights are:

The weights sum to $0.665 + 0.245 + 0.090 = 1.000$, as a softmax must. The attention-weighted aggregate (taking $W = 1$, $\sigma$ identity) is $6.65 + 0.98 + 0.09 = 7.72$ — dominated by neighbour 1, whose higher learned score earned it two-thirds of the attention. Contrast a plain mean, $(10 + 4 + 1)/3 = 5.0$: GAT’s learned weighting pulled the result sharply toward the neighbour the model decided was important.

Attention is the same idea you have met along the temporal axis — assigning weights to others and summing accordingly. The heatmap below shows attention distributed over time; a GAT does the exact same trick over a node’s neighbours: a learned weight per relationship, softmaxed to sum to one, with the high-weight links dominating the aggregate.

When to use it

Use GAT when neighbours are genuinely unequal in importance and that importance is learnable — credit-risk graphs where a single dominant counterparty drives contagion, heterogeneous transaction networks, fraud rings where a few edges carry the signal. It shines when degree-normalisation (GCN) or uniform sampling (SAGE) wrongly treat a critical edge like any other. Skip it when your effective sample size is tiny, the graph is nearly homogeneous, or a cheaper model already clears the bar — the extra parameters are a tax you pay in data you may not have.

Temporal / dynamic GNNs — when the graph moves

Analogy. A static GNN on a financial network is a single long-exposure photo of a busy intersection: every car that ever passed is smeared into one ghostly blur, and you cannot tell rush hour from 3 a.m. A temporal GNN is a video — a sequence of crisp frames — so you can see the lending network tighten before a crisis and slacken after, instead of averaging the panic and the calm into one meaningless grey.

Definition. Temporal (dynamic) GNNs model a graph that changes over time. There are two broad families:

• Snapshot-based (discrete-time). Slice time into windows, build one graph per window, embed each with a GNN, then run a sequence model across the snapshots — a GRU, LSTM, or temporal attention. Schematically, a per-slice embedding feeds a recurrence:

$z_v^{(t)} = \text{GNN}\big(G^{(t)}\big)_v, \qquad h_v^{(t)} = \text{GRU}\big(h_v^{(t-1)},\ z_v^{(t)}\big).$

• Continuous-time (event-based). Skip the snapshots and ingest timestamped events directly — each new edge (a loan at 14:32, a transfer at 14:33) updates the embeddings of the nodes it touches as it arrives. This preserves exact timing and irregular spacing that fixed snapshots throw away.

This is the deliberate marriage of two axes you have been building all along: the relational axis (a GNN, summarising who is connected to whom) and the temporal axis (an RNN or attention, summarising how things evolve), the same temporal machinery from Deep Learning for Market Data, now stacked on top of message passing.

Illustrative example — a 5-day lending sequence. Track one bank’s interbank borrowing neighbours over a week. Each day is a snapshot $G^{(t)}$; we embed it (say with GraphSAGE) into a per-day vector $z^{(t)}$, then a GRU carries state forward:

A static GNN on the time-averaged graph would report something like ”≈ 7.4 average edges, embedding ≈ 0.46” — a single number that completely hides the Monday-to-Friday collapse from 12 edges to 3. The temporal model sees the trajectory (0.20 → 0.22 → 0.41 → 0.68 → 0.79) and can flag the accelerating withdrawal of funding as a trend, which is precisely the signal a risk desk cares about.

When to use it

Use a temporal GNN whenever the network’s topology itself evolves and that evolution carries signal — funding-stress propagation, supply-chain disruption spreading link by link, fraud rings that assemble and dissolve, correlation graphs that regime-shift. Choose snapshot + sequence model when events naturally bucket into periods (daily positions, weekly exposures) and you want simplicity; choose continuous-time when exact timing and irregular event spacing matter (high-frequency settlement flows, intraday transfers). If the graph genuinely does not change over your horizon, a static GNN is simpler and fine — but in finance, “the graph doesn’t change” is a hypothesis to test, not assume.

The whole lineage on one table

The four architectures laid side by side — the comparison to keep when you choose:

Recap

You walked the lineage from a fixed recipe to a moving target. GCN averages neighbours with a degree-normalised rule nobody learned and the whole graph must be present — clean, cheap, transductive. GraphSAGE samples a handful of neighbours, learns the aggregator, concatenates self with neighbours, and earns the property that matters most in a growing financial network: it is inductive, embedding banks and wallets it never saw in training. GAT lets each node learn, via a softmax over scores, which neighbours deserve its attention — powerful where a dominant creditor truly matters, but parameter-hungry and not the free explanation it looks like. And temporal GNNs admit the graph itself moves, marrying the relational axis to the temporal one so a funding collapse reads as a trajectory, not a smeared average.