International Conference on Information and Knowledge Technology

Home / پانزدهمین کنفرانس بین المللی فناوری اطلاعات و دانش

Knowledge Graph Based Retrieval-Augmented Generation for Multi-Hop Question Answering Enhancement

Authors :

Mahdi Amiri Shavaki¹ Pouria Omrani² Ramin Toosi³ Mohammad Ali Akhaee⁴

1- دانشکده برق و کامپیوتر دانشگاه تهران 2- دانشگاه صنعتی خواجه نصیرالدین طوسی 3- دانشکده برق و کامپیوتر دانشگاه تهران 4- دانشکده برق و کامپیوتر دانشگاه تهران

Keywords :

Graph RAG،Generative AI،LLM،Multi-hop QA،NLP

Abstract :

Multi-hop question answering (QA), which requires integrating information from multiple sources, poses significant challenges in natural language processing. Existing methods often struggle with effective retrieval across documents, leading to incomplete or inaccurate answers. Building upon Graph-based Retrieval-Augmented Generation (Graph RAG), we enhance multi-hop QA by leveraging structured knowledge graphs. Specifically, we construct individual knowledge graphs for each document, where entities are represented as nodes and the relationships between them as edges enriched with contextual properties. These individual graphs are then seamlessly integrated into a comprehensive, unified graph that captures cross-document relationships. Our method improves retrieval by utilizing vector embeddings of these graph relations, enabling more effective multi-hop reasoning across the interconnected data. To evaluate our approach, we assembled a dataset of 500 documents paired with 296 multi-hop questions requiring cross-document information retrieval. Our contributions include developing a novel graph-based retrieval mechanism that leverages vector embeddings of graph relations within the Graph RAG framework, and assembling a comprehensive dataset for multi-hop QA. Comparative experiments show that our enhanced Graph RAG method significantly outperforms the baseline in factual accuracy and semantic similarity, as measured by the RAGAS framework. Additionally, an LLM-based evaluator highlights our method's superior performance in answer comprehensiveness, empowerment, and directness.