THOR: Inductive Link Prediction over Hyper-Relational Knowledge Graphs

Knowledge graphs (KGs) have become a key ingredient supporting a variety of applications. Beyond the traditional triplet representation of facts where a relation connects two entities, modern KGs observe an increasing number of hyper-relational facts, where an arbitrary number of qualifiers associated with a triplet provide auxiliary information to further describe the rich semantics of the triplet, which can effectively boost the reasoning performance in link prediction tasks. However, existing link prediction techniques over such hyper-relational KGs (HKGs) mostly focus on a transductive setting, where KG embedding models are learned from the specific vocabulary of a given KG and subsequently can only make predictions within the same vocabulary, limiting their generalizability to previously unseen vocabularies. Against this background, we propose THOR, an inducTive link prediction technique for Hyper-relational knOwledge gRaphs. Specifically, we first introduce both relation and entity foundation graphs, modeling their fundamental inter- and intra-fact interactions in HKGs, which are agnostic to any specific relations and entities. Afterward, THOR is designed to learn from the two foundation graphs with two parallel graph encoders followed by a transformer decoder, which supports efficient masked training and fully-inductive inference. We conduct a thorough evaluation of THOR in hyper-relational link prediction tasks on 12 datasets with different settings. Results show that THOR outperforms a sizable collection of baselines, yielding 66.1%, 55.9%, and 20.4% improvement over the best-performing rule-based, semi-inductive, and fully-inductive techniques, respectively. A series of ablation studies also reveals our key design factors capturing the structural invariance transferable across HKGs for inductive tasks.

💡 Research Summary

The paper addresses the growing need for inductive link prediction on hyper‑relational Knowledge Graphs (HKGs), where each fact consists of a primary triple (head, relation, tail) plus an arbitrary set of qualifier key‑value pairs. Existing methods are largely transductive: they learn embeddings for a fixed set of entities and relations and can only predict within that closed vocabulary. This limits their applicability as real‑world KGs continuously evolve with new entities and relations.

THOR (Inductive Link Prediction over Hyper‑relational Knowledge Graphs) introduces a novel two‑graph abstraction that captures the fundamental, position‑wise interactions among facts without relying on specific entity or relation identifiers.

1. Relation Foundation Graph – Nodes are relations (including primary relations and qualifier keys). Edges encode six interaction types: head‑to‑head, head‑to‑tail, tail‑to‑head, tail‑to‑tail (capturing co‑occurrence of primary relations across facts), and relation‑to‑key / key‑to‑relation (capturing how qualifiers attach to the primary relation).
2. Entity Foundation Graph – Nodes are entities (heads, tails, and qualifier values). Edges encode seven interaction types: head‑to‑tail, tail‑to‑head (the primary triple), and all permutations involving qualifier values (head‑to‑value, value‑to‑head, tail‑to‑value, value‑to‑tail, value‑to‑value).

Both graphs are agnostic to the actual symbols; they only model the structural pattern of how positions interact. This design enables the model to transfer knowledge across completely unseen vocabularies, supporting the hardest fully‑inductive scenario where training and inference graphs share no entities or relations.

For representation learning, THOR employs Neural Bellman‑Ford Networks (NBFNet) as encoders for each foundation graph. NBFNet uses a labeling trick: for a given query fact with a masked entity, all nodes participating in that query are initialized with a uniform “1” label, while other nodes receive zero. This yields conditional node embeddings that are specific to the query, rather than static embeddings tied to particular symbols. The message‑passing scheme follows the original NBFNet formulation, learning separate parameters for each edge type (the interaction types defined above). Consequently, the encoders learn interaction‑level parameters rather than entity‑ or relation‑specific vectors, which is essential for inductive generalisation.

The conditional embeddings from the two encoders are concatenated and fed into a Transformer‑based decoder. The decoder adopts an edge‑biased self‑attention mechanism: each attention head incorporates learnable bias terms that reflect the edge type between any pair of elements (e.g., head‑to‑value bias). This allows the model to capture the positional semantics inside a hyper‑relational fact. Training uses a masked language modeling style objective: the