Knowledge Graph Completion Based on Bipartite Graphs and Attention Mechanism

0
0
0

Knowledge Graph Completion Based on Bipartite Graphs and Attention Mechanism

Introduction

Knowledge graphs (KGs) have become a fundamental tool for representing structured information in various applications, including question answering, recommendation systems, and information retrieval. A KG typically consists of triples in the form (head entity, relation, tail entity), which describe relationships between entities or concepts. Despite their widespread use, even large-scale KGs such as Wikidata and Google KG suffer from incompleteness, limiting their effectiveness in downstream tasks. Knowledge graph completion (KGC) aims to address this issue by predicting missing links in the KG, thereby enhancing its coverage and utility.

Traditional KGC methods primarily fall into two categories: knowledge graph embedding (KGE) techniques and neural network-based approaches. While KGE methods, such as TransE and its variants, efficiently learn low-dimensional representations of entities and relations, they often struggle with complex relational patterns (e.g., one-to-many or many-to-one relations). On the other hand, neural network-based models, particularly those leveraging graph neural networks (GNNs), have shown promise in capturing structural information. However, existing GNN-based models predominantly focus on entity neighborhood information while neglecting the potential structural dependencies among relations. Additionally, these models often employ static attention mechanisms, which treat all neighboring information equally, leading to suboptimal performance.

To overcome these limitations, this paper introduces a novel model called Capturing Global Information Based on Bipartite and Attention Mechanism (CGIBAM). The model leverages bipartite graphs and an attention mechanism to capture both entity neighborhood information and relational structural dependencies. By integrating these components, CGIBAM enhances the model’s ability to learn global structural patterns, thereby improving KGC performance.

Related Work

Knowledge Graph Completion Models

Existing KGC models can be broadly classified into three categories:

  1. Translation-Based Models: These models, such as TransE, TransH, and TransR, represent entities and relations as vectors in a low-dimensional space. They use distance-based scoring functions to measure the plausibility of triples. While effective for simple relations, these models struggle with complex relational patterns.

  2. Semantic Matching Models: Approaches like ComplEx and QuatE use scoring functions to measure the compatibility between entity pairs and relations. These models often operate independently on triples, ignoring contextual neighborhood information.

  3. Neural Network-Based Models: GNN-based models, such as R-GCN and CompGCN, incorporate relational information into entity embeddings. However, they primarily focus on entity-centric structures and fail to capture relational dependencies effectively.

Attention Mechanisms in KGC

Attention mechanisms have been widely adopted in deep learning to dynamically weigh the importance of different features. In KGC, attention helps identify relevant interactions between entities and relations. However, traditional static attention mechanisms lack adaptability, leading to performance bottlenecks. Recent advancements, such as multi-head attention, allow models to focus on different aspects of the input, improving feature extraction.

Model Architecture

The proposed CGIBAM model follows an encoder-decoder architecture, consisting of three main components:

  1. Bipartite Graph Construction
  2. Encoder Module
  3. Decoder Module

Bipartite Graph Construction

To capture both entity neighborhood information and relational structural dependencies, CGIBAM constructs two subgraphs:

  1. Entity-Centric Subgraph (G_ef): This undirected graph treats entities as nodes and connects them based on their relationships in the original KG. The adjacency matrix is constructed by iterating over all triples, ensuring bidirectional connections between head and tail entities.

  2. Relation-Centric Subgraph (G_rf): This subgraph treats entity-relation pairs as nodes, capturing potential dependencies between relations. For example, given a triple (John Wilson, born in, Sydney), the model identifies relations connected to “Sydney” (e.g., “city of”) and constructs a new triple (born in, Sydney, city of). This helps capture implicit relational patterns that may aid in predicting missing links.

Encoder Module

The encoder processes the bipartite graphs using two GNN layers:

  1. Entity Embedding Update: The first GNN layer aggregates neighborhood information for each entity, generating updated embeddings.
  2. Relation Embedding Update: The second GNN layer processes the relation-centric subgraph to refine relation embeddings.

The updated entity and relation embeddings are then combined and processed through a multi-head attention mechanism. This mechanism dynamically assigns weights to different interactions, allowing the model to focus on the most relevant features.

Decoder Module

The decoder employs the TuckER scoring function to evaluate the plausibility of candidate triples. Given an input triple (h, r, t), the model computes a score indicating the likelihood of the triple being valid. A sigmoid activation function is applied to produce the final prediction.

Experiments

Datasets

The model was evaluated on three benchmark datasets:

  1. FB15K-237: A subset of Freebase with 14,541 entities and 237 relations.
  2. WN18RR: A subset of WordNet with 4,094 entities and 11 relations.
  3. NELL995: A dataset from the NELL system with 63,917 entities and 198 relations.

Evaluation Metrics

Performance was measured using:
• Mean Reciprocal Rank (MRR): The average reciprocal rank of correct entities.

• Hits@k: The proportion of correct entities ranked in the top k predictions (k=1, 3, 10).

Results

CGIBAM outperformed baseline models across all datasets:

• On FB15K-237, MRR improved by 5.1%, Hits@1 by 2.6%, Hits@3 by 7%, and Hits@10 by 8.8%.

• On WN18RR, MRR improved by 0.1%, Hits@1 by 0.2%, Hits@3 by 0.6%, and Hits@10 by 1.9%.

• On NELL995, MRR improved by 3.4%, Hits@1 by 0.9%, Hits@3 by 2.4%, and Hits@10 by 2.2%.

Ablation Study

To validate the contributions of each component, ablation studies were conducted:

  1. Removing Multi-Head Attention (CGIBAM-att): Performance dropped significantly, confirming the importance of dynamic feature weighting.
  2. Removing Bipartite Graphs (CGIBAM-bip): The model’s ability to capture relational dependencies diminished, leading to lower accuracy.

Impact of Learning Rate

Experiments with different learning rates (λ) revealed:
• For FB15K-237 and NELL995, λ=0.01 yielded the best results.

• For WN18RR, λ=0.001 was optimal due to the dataset’s smaller size and fewer relations.

Conclusion

The CGIBAM model introduces a novel approach to KGC by leveraging bipartite graphs and attention mechanisms to capture both entity neighborhood information and relational structural dependencies. Experimental results demonstrate significant improvements over existing methods, particularly in complex, multi-relational datasets. Future work will explore integrating temporal information and further optimizing the model’s generalization capabilities.

doi.org/10.19734/j.issn.1001-3695.2024.06.0186

Was this helpful?

0 / 0