[Purpose/Significance] This study focuses on employing a multi-agent collaborative framework to decompose and execute complex tasks, proposing a graph retrieval-augmented generative Q&A system that integrates knowledge graphs with large language models within such a framework. The system aims to address common limitations of traditional Q&A systems, such as inefficient retrieval of complex queries, low credibility of responses, and insufficient accuracy. [Method/Process] A lightweight library QA system was constructed using data from the Peking University Library digital platform. First, KG construction was achieved through entity recognition, knowledge extraction, and storage in a Neo4j graph database. Subsequently, the large language model DeepSeek-R1 was locally deployed and linked to the KG. Finally, a graph-retrieval-augmented generative library QA system was built based on the multi-agent collaborative framework, where multiple agents collaborate to decompose and execute user questions. [Result/Conclusion] Testing using a combination of human evaluation and automatic evaluation demonstrated that the graph-retrieval-augmented generative QA system incorporating the multi-agent collaborative framework exhibits superior capability in solving complex problems compared to the system without it. Improvements of 11.7%, 11.4%, and 15.7% were observed on the BLEU-4, ROUGE-L, and METEOR metrics, respectively. The system exhibits strong capabilities in decomposing and resolving complex queries, while enhancing the accuracy and credibility of responses, thereby addressing the limitations of graph retrieval-augmented generative Q&A systems.
[1] 胡潇戈, 戚越, 王玉琦, 等. 面向智能问答的图书馆参考咨询知识库体系设计及构建[J]. 图书情报知识, 2019(5): 101-108, 119. (HU X G, QI Y, WANG Y Q, et al. Design and construction of intelligent question-answer knowledge base for library reference and consultancy[J]. Documentation, information & knowledge, 2019(5): 101-108, 119.)
[2] 吉万明. 智慧图书馆新技术应用的问题及对策[J]. 中国管理信息化, 2024, 27(22): 178-180. (JI W M. Problems and countermeasures of new technology application in smart libraries[J]. China management informatization, 2024, 27(22): 178-180.)
[3] 国务院办公厅. 国务院办公厅印发《关于推动文化高质量发展的若干经济政策》的通知[EB/OL]. [2025-08-30]. https://www.gov.cn/zhengce/content/202501/content_7000958.htm. (General Office of the State Council. Notice on issuing the “Several Economic Policies for Promoting High-Quality Cultural Development” [EB/OL]. [2025-08-30]. https://www.gov.cn/zhengce/content/202501/content_7000958.htm.)
[4] 王守会, 覃飙. 知识库问答系统研究进展[J]. 小型微型计算机系统, 2021, 42(9): 1793-1801. (WANG S H, QIN B. Research progress of knowledge base question answering[J]. Journal of Chinese computer systems, 2021, 42(9): 1793-1801.)
[5] 闫悦, 郭晓然, 王铁君, 等. 问答系统研究综述[J]. 计算机系统应用, 2023, 32(8): 1-18. (YAN Y, GUO X R, WANG T J, et al. Survey on question answering system research[J]. Computer systems & applications, 2023, 32(8): 1-18.)
[6] BEAULIEU-JONES B R, BERRIGAN M T, SHAH S, et al. Evaluating capabilities of large language models: performance of GPT-4 on surgical knowledge assessments[J]. Surgery, 2024, 175(4): 936-942.
[7] YOHANNES H M, AMAGASA T. Named-entity recognition for a low-resource language using pre-trained language model[C]//HONG J. Proceedings of the 37th ACM/SIGAPP symposium on applied computing. New York: Association for Computing Machinery, 2022: 837-844.
[8] WANG C, LIU P, ZHANG Y. Can generative pre-trained language models serve as knowledge bases for closed-book QA?[C]//CHENG Q Z. Proceedings of the 59th annual meeting of the association for computational linguistics and the 11th international joint conference on natural language processing. New York: Association for Computational Linguistics, 2021: 3241-3251.
[9] FENG T, HE L. RGR-KBQA: generating logical forms for question answering using knowledge-graph-enhanced large language model[C]//RAMBOW O. Proceedings of the 31st International Conference on Computational Linguistics. New York: Association for Computational Linguistics, 2025: 3057-3070.
[10] HUANG F, DENG Y, ZHANG C, et al. KOSA: KO enhanced salary analytics based on knowledge graph and LLM capabilities[C]//DAI H. 2023 IEEE International Conference on Data Mining Workshops (ICDMW). New York: IEEE, 2023: 499-505.
[11] SCHAEFFER R, MIRANDA B, KOYEJO S. Are emergent abilities of large language models a mirage?[J]. Advances in neural information processing systems, 2023, 36: 55565-55581.
[12] CHEN Y, SHEN Y. Temporal knowledge graph link prediction using synergized large language models and temporal knowledge graphs[C]//ZHANG H. International Conference on Neural Computing for Advanced Applications. Singapore: Springer Nature Singapore, 2024: 33-45.
[13] WENG X, WANG Y, WENG S, et al. Bibliometric analysis of large language model artificial intelligence based on knowledge graph[C]//FANG B. 2024 IEEE 9th International Conference on Data Science in Cyberspace (DSC). New York: IEEE, 2024: 428-434.
[14] CHO N, SRISHANKAR N, CECCHI L, et al. FISHNET: financial intelligence from sub-querying, harmonizing, neural-conditioning, expert swarms, and task planning[C]//MEHTA D. Proceedings of the 5th ACM International Conference on AI in Finance. New York: Association for Computing Machinery, 2024: 591-599.
[15] ALAM M, BUSCALDI D, COCHEZ M, et al. Workshop on deep learning and large language models for knowledge graphs (DL4KG)[C]//BAEZA-YATES R. Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. New York: Association for Computing Machinery, 2024: 6704-6705.
[16] ZHU Z, QI G, SHANG G, et al. Enhancing large language models with knowledge graphs for robust question answering[C]//LIU Y. 2024 IEEE 30th International Conference on Parallel and Distributed Systems (ICPADS). New York: IEEE, 2024: 262-269.
[17] KOOHBORFDHAGHIGHI S, DE GEYTER G, KALINER E. Unlocking the power of LLM-based question answering systems: enhancing reasoning, insight, and automation with knowledge graphs[C]//SIARRY P. International Conference on Intelligent Systems Design and Applications. New York: Springer Nature Switzerland, 2023: 156-171.
[18] AMIRKHANI A, BARSHOOI A H. Consensus in multi-agent systems: a review[J]. Artificial intelligence review, 2022, 55(5): 3897-3935.
[19] XU Z, LI J, ZHANG W. Large language model and knowledge graph entangled logical reasoning[C]//WU X. 2024 IEEE International Conference on Knowledge Graph (ICKG). New York: IEEE, 2024: 432-439.
[20] LIU Y H, SONG G, NI W J, et al. Event-driven multi-agent evacuation based on reinforcement learning[C]//ZENG J. Third International Conference on Intelligent Computing and Human-Computer Interaction (ICHCI 2022). New York: SPIE, 2023, 12509: 533-538.
[21] ZHAO J, MA Z, ZHAO H, et al. Self-consistency, extract and rectify: knowledge graph enhance large language model for electric power question answering[C]//HUANG D S. International Conference on Intelligent Computing. New York: Springer Nature Singapore, 2024: 493-504.
[22] LI W, QI X, ZHAO Q, et al. Knowledge graph-based credibility evaluation method for electric grid large language model knowledge question-answering[C]//WANG Y. Proceedings of the 20237th International Conference on Electronic Information Technology and Computer Engineering. New York: Association for Computing Machinery, 2023: 754-759.
[23] GUO Q, CAO S, YI Z. A medical question answering system using large language models and knowledge graphs[J]. International journal of intelligent systems, 2022, 37(11): 8548-8564.
[24] TAO W, ZHU H, TAN K, et al. FinQA: a training-free dynamic knowledge graph question answering system in finance with LLM-based revision[C]//BRAZDIL P. Joint European Conference on Machine Learning and Knowledge Discovery in Databases. New York: Springer Nature Switzerland, 2024: 418-423.
[25] 文化和旅游部公共服务司. 文化和旅游部关于印发《“十四五”公共文化服务体系建设规划》的通知[EB/OL].[2025-08-30].https://zwgk.mct.gov.cn/zfxxgkml/ggfw/202106/t20210623_925879.html. (Public Service Department of Ministry of Culture and Tourism of the People’s Republic of China. Ministry of Culture and Tourism. Notice on issuing the “14th Five-Year Plan for Public Cultural Service System Construction” [EB/OL]. [2025-08-30]. https://zwgk.mct.gov.cn/zfxxgkml/ggfw/202106/t20210623_925879.html.)
[26] 赵浜, 曹树金. 生成式AI大模型结合知识库与AI Agent开展知识挖掘的探析[J]. 图书情报知识, 2025, 42(4): 88-101. (ZHAO B, CAO S J. Exploring knowledge mining using large language model combined with knowledge base and AI Agent[J]. Documentation, Information & Knowledge, 2025, 42(4): 88-101.)
[27] 王帅, 何文春, 王甫棣, 等. 大语言模型融合知识图谱与向量检索的问答系统[J]. 科学技术与工程, 2024, 24(32): 13902-13910. (WANG S, HE W C, WANG F D, et al. Question answering system based on large language model integrating knowledge graph and vector retrieval[J]. Science technology and engineering, 2024, 24(32): 13902-13910.)
[28] MALDONADO D, CRUZ E, TORRES J A, et al. Multi-agent systems: a survey about its components, framework and workflow[J]. IEEE Access, 2024(12): 80950-80975.
[29] 潘雨亭, 柯平. 生成式AI在书目工作中的应用效果及启示：基于文心一言和DeepSeek的实验分析[J].现代情报, 2025, 45(6): 26-33. (PAN Y T, KE P. Application effects and implications of generative AI in bibliographic work——an experimental analysis based on ERNIE Bot and DeepSeek[J]. Journal of modern information, 2025, 45(6): 26-33.)
[30] 叶蕾, 张宇迪, 杨旭华. 利用知识图谱的多跳可解释问答[J]. 小型微型计算机系统, 2024, 45(8): 1869-1877. (YE L, ZHANG Y D, YANG X H. Multi-hop interpretable question answering using knowledge graph[J]. Journal of Chinese computer systems, 2024, 45(8):1869-1877.)
[31] 冯拓宇, 李伟平, 郭庆浪, 等. 大语言模型增强的知识图谱问答研究进展综述[J]. 计算机科学与探索, 2024, 18(11): 2887-2900. (FENG T Y, LI W P, GUO Q L, et al. Overview of knowledge graph question answering enhanced by large language models[J]. Journal of frontiers of computer science and technology, 2024, 18(11): 2887-2900.
