Knowledge Graph Link Prediction Based on Multi-Feature Extraction and Contrastive Learning

Introduction

Knowledge graphs (KGs) have become a fundamental tool for representing structured knowledge, capturing relationships between entities in various domains such as information retrieval, question answering, and recommendation systems. However, real-world knowledge graphs are often incomplete, with many missing links between entities. For instance, in FreeBase, approximately 70% of person entities lack birthplace information, and 99% lack ethnicity details. Link prediction, a core task in knowledge graph completion (KGC), aims to infer missing relationships or entities based on existing facts. Traditional methods, such as TransE, rely on simple vector operations to model relationships but struggle with complex relational patterns like one-to-many, many-to-one, and many-to-many relationships.

Recent advancements in deep learning have introduced more sophisticated approaches, including graph neural networks (GNNs) and convolutional neural networks (CNNs), to improve link prediction. However, many existing methods suffer from limitations such as shallow feature extraction, insufficient consideration of node interactions, and low-quality negative sampling. To address these challenges, this paper proposes a novel link prediction method called MFCL (Multi-Feature extraction and Contrastive Learning), which integrates multi-feature aggregation, deep feature extraction, and contrastive learning to enhance prediction accuracy.

Methodology

Task Definition

A knowledge graph is represented as ( G = (E, R, T) ), where ( E ) and ( R ) denote the sets of entities and relations, respectively, and ( T ) consists of factual triples ( (h, r, t) ), where ( h ) and ( t ) are head and tail entities, and ( r ) is their relationship. The link prediction task involves predicting missing entities or relations. For example, given ( (h, r, ?) ), the model predicts the most likely tail entity ( t ).

Overall Framework

The MFCL model consists of three main components:

Multi-Feature Aggregation Module: Captures node features from multiple perspectives by combining graph attention networks (GAT) and weighted graph convolutional networks (WGCN).
Deep Feature Extraction Module: Uses deep convolutional networks to model complex interactions between entities and relations.
Negative Triplet Optimization Module: Enhances negative sampling quality through contrastive learning and a self-scoring mechanism.

Multi-Feature Aggregation Module

This module extracts node features by considering both entity importance and relation importance.

Entity Importance Embedding via GAT
GAT computes attention weights between nodes to determine their relative importance. For a target node ( ei ), the attention coefficient ( alpha{ij} ) measures the influence of neighbor ( e_j ). The final node representation aggregates features from multiple attention heads, allowing the model to focus on the most relevant neighbors.

Relation Importance Embedding via WGCN
WGCN extends traditional GCN by assigning adaptive weights to different relation types. Each relation type ( t ) has a learnable weight ( varphi_t ), which adjusts during training to reflect the importance of different relationships. The node embedding is updated by aggregating weighted neighbor features, ensuring that the model captures diverse relational patterns.

The outputs of GAT and WGCN are combined to form a comprehensive node representation that integrates both structural and relational information.

Deep Feature Extraction Module

Shallow convolutional networks in existing models often fail to capture deep semantic interactions between entities and relations. To address this, MFCL employs a deep CNN architecture that processes entity-relation pairs as 2D matrices. Multiple convolutional filters extract global and transitional features, while residual connections and pooling layers enhance feature stability.

The module ensures that entity-relation interactions are modeled at multiple levels, improving the model’s ability to generalize across complex relational patterns.

Negative Triplet Optimization Module

Traditional negative sampling methods randomly replace entities in positive triples, leading to low-quality negative samples that hinder model training. MFCL introduces two key improvements:

Contrastive Learning-Based Sampling
Within each training batch, negative samples are selected from other triples in the same batch (in-batch negative sampling). This approach increases training efficiency while maintaining diversity in negative samples.

Self-Scoring Mechanism
A scoring function evaluates candidate negative triples based on cosine similarity. Only samples with scores within a dynamically adjusted range are retained, ensuring that negative samples are neither too easy nor too hard for the model.

Triplet Feature Enhancement
To further improve negative sample quality, MFCL mixes features of negative entities with those of high-scoring positive entities. This adaptive blending ensures that negative samples contain meaningful semantic information, making them more challenging and informative for the model.

The final loss function incorporates a margin-based contrastive loss, encouraging the model to distinguish between true and false triples effectively.

Experiments

Datasets and Evaluation Metrics

Experiments were conducted on two standard datasets: FB15k-237 and WN18RR. FB15k-237 contains 14,541 entities and 237 relations, while WN18RR has 40,943 entities and 11 relations. Performance was evaluated using Hits@1, Hits@3, Hits@10, and Mean Reciprocal Rank (MRR).

Baseline Models

MFCL was compared against several state-of-the-art models, including:
• TransE: A translation-based model that uses simple vector operations.

• SACN: Combines WGCN with Conv-TransE for improved feature extraction.

• PairRE: Uses relation-specific embeddings to handle complex relations.

• KG-R3: A retrieval-based model that focuses on contextual subgraphs.

• HyGGE: Leverages hyperbolic geometry to capture hierarchical structures.

Results

MFCL outperformed baseline models across most metrics. On FB15k-237, it achieved a Hits@10 of 0.593 and an MRR of 0.401, surpassing MetaSD by 2.2% and 1.0%, respectively. On WN18RR, MFCL achieved a Hits@10 of 0.596 and an MRR of 0.513, demonstrating robustness even in datasets with fewer relation types.

Ablation Studies

Removing any of the three main modules led to performance degradation:
• Without multi-feature aggregation, Hits@10 dropped by 6.4% on FB15k-237.

• Without deep feature extraction, MRR decreased by 1.7%.

• Without negative triplet optimization, Hits@10 declined by 2.4%.

These results confirm that each module contributes significantly to the model’s success.

Hyperparameter Analysis

Key hyperparameters were tuned to optimize performance:
• GAT Layers: Two layers provided the best balance between feature depth and computational efficiency.

• Embedding Dimension: A dimension of 150 achieved optimal results before overfitting occurred.

• Learning Rate: A value of 2e-4 ensured stable convergence.

• Temperature Coefficient: A value of 0.005 maximized contrastive learning effectiveness.

Case Study

MFCL was tested on real-world triples from FB15k-237. For example, given ( (England, contains, ?) ), MFCL correctly predicted “London” as the top result, while other models like ConvE and SACN produced less accurate rankings. This demonstrates MFCL’s ability to leverage multi-feature representations for precise predictions.

Conclusion

This paper presents MFCL, a novel link prediction framework that integrates multi-feature aggregation, deep feature extraction, and contrastive learning. By dynamically capturing node interactions and optimizing negative sampling, MFCL achieves state-of-the-art performance on standard benchmarks. Future work will explore global graph information to further enhance node embeddings.

doi.org/10.19734/j.issn.1001-3695.2024.06.0238

Was this helpful?

0 / 0