A transformer framework for generating context-aware knowledge graph paths
Publication Type
Journal Article
Publication Date
7-2023
Abstract
Contextual Path Generation (CPG) refers to the task of generating knowledge path(s) between a pair of entities mentioned in an input textual context to determine the semantic connection between them. Such knowledge paths, also called contextual paths, can be very useful in many advanced information retrieval applications. Nevertheless, CPG involves several technical challenges, namely, sparse and noisy input context, missing relations in knowledge graphs, and generation of ill-formed and irrelevant knowledge paths. In this paper, we propose a transformer-based model architecture. In this approach, we leverage a mixture of pre-trained word and knowledge graph embeddings to encode the semantics of input context, a transformer decoder to perform path generation controlled by encoded input context and head entity to stay relevant to the context, and scaling methods to sample a well-formed path. We evaluate our proposed CPG models derived using the above architecture on two real datasets, both consisting of Wikinews articles as input context documents and ground truth contextual paths, as well as a large synthetic dataset to conduct larger-scale experiments. Our experiments show that our proposed models outperform the baseline models, and the scaling methods contribute to better quality contextual paths. We further analyze how CPG accuracy can be affected by different amount of context data, and missing relations in the knowledge graph. Finally, we demonstrate that an answer model for knowledge graph questions adapted for CPG could not perform well due to the lack of an effective path generation module.
Keywords
Knowledge graph, Contextual path generation, Information retrieval, Language model
Discipline
Databases and Information Systems | Graphics and Human Computer Interfaces
Research Areas
Data Science and Engineering
Publication
Applied Intelligence
First Page
1
Last Page
28
ISSN
0924-669X
Identifier
10.1007/s10489-023-04588-3
Publisher
Springer
Citation
LO, Pei-Chi and LIM, Ee-peng.
A transformer framework for generating context-aware knowledge graph paths. (2023). Applied Intelligence. 1-28.
Available at: https://ink.library.smu.edu.sg/sis_research/8026
Copyright Owner and License
Authors
Additional URL
https://doi.org/10.1007/s10489-023-04588-3