Geometric Priors in Graph Neural Networks
Published:
The Geometry of Message Passing
Graph Neural Networks operate by message passing — nodes aggregate information from their neighbors:
\[h_v^{(l+1)} = \sigma\left(W^{(l)} \sum_{u \in \mathcal{N}(v)} \frac{h_u^{(l)}}{\sqrt{d_v d_u}}\right)\]But this formulation assumes Euclidean geometry — it uses vector addition, which is meaningful only in flat space.
Curvature-Aware Aggregation
What if the graph has intrinsic curvature? For negatively curved (hyperbolic) graphs, we should aggregate in hyperbolic space:
\[h_v^{(l+1)} = \exp_{x_v}\left(\sum_{u \in \mathcal{N}(v)} w_{vu} \log_{x_v}(h_u^{(l)})\right)\]where \(\exp\) and \(\log\) are the exponential and logarithmic maps of the manifold.