International Conference on Information and Knowledge Technology

Home / پانزدهمین کنفرانس بین المللی فناوری اطلاعات و دانش

KGLM-QA: A Novel Approach for Knowledge Graph-Enhanced Large Language Models for Question Answering

Authors :

Alireza Akhavan safaei¹ Pegah Saboori² Reza Ramezani³ Mohammadali Nematbakhsh⁴

1- دانشگاه اصفهان 2- دانشگاه اصفهان 3- دانشگاه اصفهان 4- دانشگاه اصفهان

Keywords :

Large Language Models،Knowledge graph،Question Answering،Retrieval augmented generation

Abstract :

Large language models excel in various natural language processing tasks but often struggle with knowledge-intensive queries, particularly those involve rare entities or require precise factual information. This paper presents a novel framework that enhances capabilities of an LLM-based question answering system by incorporating structured knowledge from knowledge graphs. Our approach employs entity extraction, semantic similarity scoring, and adaptive graph exploration to efficiently navigate and extract relevant information from knowledge graphs. The core of the presented solution is a knowledge graph-enhanced language model process that iteratively refines subgraph exploration and answer generation, complemented by a fallback mechanism for robustness across diverse question types. Experiments on location-based questions from the Entity Questions dataset demonstrate significant improvements in the quality of responses. Using the Gemini 1.5 Flash model, our system achieved an accuracy increase from 36% to 71% for partially correct answers and from 22% to 69% for exactly correct answers, as evaluated by human assessors. This approach offers a promising direction for developing more reliable and accurate question answering systems, particularly for queries involving long-tail entities or specific factual knowledge.