研究论文

图书馆场景下知识服务智能问答机器人人智交互效果评价

  • 罗然 ,
  • 王晰巍 ,
  • 李姝璇
展开
  • 1 吉林大学商学与管理学院 长春 130012;
    2 吉林大学大数据管理研究中心 长春 130012;
    3 吉林大学网络空间治理研究中心 长春 130012
罗然,助理研究员,博士;王晰巍,教授,博士,博士生导师,通信作者,E-mail:wxw_mail@163.com;李姝璇,博士研究生。

收稿日期: 2025-06-30

  修回日期: 2025-10-23

  网络出版日期: 2026-02-12

基金资助

本文系教育部人文社会科学研究一般项目“信息生态视角下人工智能生成内容人机交互风险及治理路径研究”(项目编号:24YJA870012)研究成果之一。

Evaluation of Human-AI Interaction Effectiveness in Knowledge Service Chatbots in Library Scenarios

  • Luo Ran ,
  • Wang Xiwei ,
  • Li Shuxuan
Expand
  • 1 School of Business and Management, Jilin University, Changchun 130012;
    2 Research Center for Big Data Management, Jilin University, Changchun 130012;
    3 Research Center of Cyberspace Governance, Jilin University, Changchun 130012

Received date: 2025-06-30

  Revised date: 2025-10-23

  Online published: 2026-02-12

Supported by

This work is supported by the Planning Fund Project of Humanities and Social Science Research of the Ministry of Education of the People’s Republic of China, titled “Research on Human-Computer Interaction Risks and Governance Paths of AI-Generated Content from the Perspective of Information Ecology” (Grant No. 24YJA870012).

摘要

[目的/意义]生成式AI技术正在重塑图书馆知识服务范式。图书馆场景下的知识服务智能问答机器人人智交互效果评价研究,能够助力图书馆智慧知识服务转型中AI系统的痛点定位与优化方向。[方法/过程]本研究基于信息生态理论,从信息技术、信息、信息人、信息环境4个维度构建图书馆场景下的知识服务智能问答机器人人智交互效果评价指标体系,通过网络分析法与熵值法相结合的组合赋权法确定指标权重。基于模糊综合评价法构建人智交互效果评价分析模型,通过实证研究验证指标体系与评价分析模型的可行性。[结果/结论]数据研究表明,需要从信息技术和信息两个维度提升用户交互体验,并且着重关注智能问答机器人在逻辑推理、上下文理解、移情性、便利性、拟人化等方面的提升,推动图书馆场景下的知识服务智能问答机器人从传统检索工具向具备“智商+情商”的知识服务主体转型。本研究能够为图书馆场景下智能问答系统的知识服务效能优化提供可量化的决策依据,促进图书馆优化以用户需求为中心、以智能技术为驱动、以生态协同为支撑的智慧知识服务模式。

本文引用格式

罗然 , 王晰巍 , 李姝璇 . 图书馆场景下知识服务智能问答机器人人智交互效果评价[J]. 图书情报工作, 2026 , 70(4) : 100 -115 . DOI: 10.13266/j.issn.0252-3116.2026.04.008

Abstract

[Purpose/Significance] Generative AI technology is reshaping the paradigm of library knowledge services. Evaluating the human-AI interaction effectiveness of knowledge service chatbots in library scenarios can help identify pain points and optimization directions for AI systems during libraries’ transition to smart knowledge services. [Method/Process] Based on information ecology theory, this study constructs an evaluation index system for human-AI interaction effectiveness in knowledge service chatbots in library scenarios across four dimensions: information technology, information, information people, and information environment. A combination weighting method integrating the analytic network process (ANP) and the entropy weight method is employed to determine indicator weights. A fuzzy comprehensive evaluation model is developed to assess human-AI interaction effectiveness, and empirical research validates the feasibility of the index system and the evaluation model. [Result/ Conclusion] Data analysis reveals that libraries should enhance user interaction by prioritizing improvements in the information technology and information dimensions, particularly in logical reasoning, contextual understanding, empathy, convenience, and anthropomorphism. This transformation shifts chatbots from traditional retrieval tools to knowledge service chatbots with both “IQ and EQ”. The study provides quantifiable decision-making support for optimizing the service efficacy of intelligent Q&A systems, promoting a smart knowledge service model that is user-centric, technology-driven, and supported by ecological synergy.

参考文献

[1] 李强. 政府工作报告——2025年 3月 5日在第十四届全国人民代表大会第三次会议上[EB/OL]. [2025-12-01]. https://www. gov.cn/gongbao/2025/issue_11946/202503/content_7015861. html. (LI Q. Government Work Report-delivered at the third plenary session of the 14th National People’s Congress on March 5, 2025[EB/OL]. [2025-12-01]. https://www.gov.cn/gongbao/2025/issue_11946/202503/content_7015861.html.)
[2] ZHONG T L, ZHU G X, HOU C Y, et al. The influences of ChatGPT on undergraduate students’ demonstrated and perceived interdisciplinary learning[J]. Education and information technologies, 2024, 29(17): 23577-23603.
[3] CHU H R, LIU S X. Can AI tell good stories? narrative transportation and persuasion with ChatGPT[J]. Journal of communication, 2024, 74(5): 347-358.
[4] WANG L, CHEN X Y, WANG C, et al. ChatGPT’s capabilities in providing feedback on undergraduate students’ argumentation: a case study[J]. Thinking skills and creativity, 2024, 51: 14.
[5] ARIYARATNE S, IYENGAR K P, NISCHAL N, et al. A comparison of ChatGPT-generated articles with human-written articles[J]. Skeletal radiology, 2023, 52(9): 1755-1758.
[6] 张晓林. 走向知识服务:寻找新世纪图书情报工作的生长点[J]. 中国图书馆学报, 2000(5): 30-35. (ZHANG X L. Towards knowledge services: seeking development opportunities for library and information services in the 21st century[J]. Journal of library science in China, 2000(5): 30-35.)
[7] 姜婷婷, 田慧溢, 许艳闰, 等. 人智交互体验度量:受控实验的应用及启示[J]. 图书情报知识, 2023, 40(2): 10-19. (JIANG T T, TIAN H Y, XU Y R, et al. Measuring human-AI interaction experience: implications for conducting controlled experiments[J]. Documentation, information & knowledge, 2023, 40(2): 10-19.)
[8] JANG S, LEE H, KIM Y, et al. When, what, and how should generative artificial intelligence explain to users?[J]. Telematics and informatics, 2024, 93: 102175.
[9] 白英彩. 英汉计算机技术大辞典[M]. 上海: 上海交通大学出版社, 2001. (BAI Y C. An English-Chinese computer technology dictionary[M]. Shanghai: Shanghai Jiao Tong University Press, 2001.)
[10] 钱力, 张智雄, 伍大勇, 等. 科技文献大模型:方法、框架与应用[J]. 中国图书馆学报, 2024, 50(6): 45-58. (QIAN L, ZHANG Z X, WU D Y, et al. The large language model for scientific literature: method, framework, and application[J]. Journal of library science in China, 2024, 50(6): 45-58.)
[11] 吴丹, 郭清玥, 杨馨梅, 等. 智慧图书馆场景下数据与知识双驱动方法创新研究[J]. 中国图书馆学报, 2025, 51(5): 37-53. (WU D, GUO Q Y, YANG X M, et al. Methodological innovation on dual data- and knowledge-driven approach for smart libraries scenarios[J]. Journal of library science in China, 2025, 51(5): 37- 53.)
[12] 王晰巍, 刘婷艳, 李玛莉. 直播带货平台用户信息交互效果评价研究[J]. 现代情报, 2022, 42(11): 17-26. (WANG X W, LIU T Y, LI M L. Research on user information interaction effect evaluation of live streaming platform[J]. Journal of modern information, 2022, 42(11): 17-26.)
[13] LI L L, COATES K. Academic library online chat services under the impact of artificial intelligence[J]. Information discovery and delivery, 2025, 53(2): 192-205.
[14] ABOELMAGED M, BANI-MELHEM S, AL-HAWARI M A, et al. Conversational AI Chatbots in library research: an integrative review and future research agenda[J]. Journal of librarianship and information science, 2025, 57(2): 331-347.
[15] 李立睿, 张嘉程, 邓小昭. 图书馆机器人服务用户体验影响机理研究 [J]. 图书情报工作, 2024, 68(12): 43-55. (LI L R, ZHANG J C, DENG X Z. Research on the influencing mechanism of user experience of robot services in libraries[J]. Library and information service, 2024, 68(12): 43-55.)
[16] 王娜, 李杰. 基于AHP- 熵权法的FAQ问答系统用户满意度评价研究—— 以高校图书馆问答型机器人为例[J]. 情报科学, 2023, 41(9): 164-172. (WANG N, LI J. User satisfaction evaluation of FAQ system based on AHP-entropy weight method: taking the question answering robot of university library as an example[J]. Information science, 2023, 41(9): 164-172.)
[17] 李晓旭. 我国公共图书馆老年人健康信息服务质量评价体系构建—— 基于信息生态理论视角[J]. 图书馆工作与研究, 2025(4): 5-13, 23. (LI X X. Quality evaluation system construction of elderly health information services in public libraries in China: based on the perspective of information ecology theory[J]. Library work and study, 2025(4): 5-13, 23.)
[18] 刘于蓝, 张艳丰, 马家伟, 等. 我国省级政府数据开放平台信息服务质量测度实证研究[J]. 情报科学, 2024, 42(9): 165- 177. (LIU Y L, ZHANG Y F, MA J W, et al. Empirical study on measuring the quality of information services on provincial government data open platforms in China [J]. Information science, 2024, 42(9): 165-177.)
[19] 赵英, 肖垚, 李佳, 等. 在线医疗平台信息生态系统的生态性评价指标体系研究[J]. 情报理论与实践, 2024, 47(7): 59-68. (ZHAO Y, XIAO Y, LI J, et al. Research on ecological evaluation index system of online medical platform information ecosystem[J]. Information studies: theory & application, 2024, 47(7): 59-68.)
[20] 刘泽, 邵波. 区域性高校图书馆联盟数字资源服务能力成熟度评价与实证研究[J]. 图书情报工作, 2025, 69(18): 16- 29. (LIU Z, SHAO B. Evaluation and empirical research on the maturity of digital resource service capability of the regional university library consortium[J]. Library and information service, 2025, 69(18): 16-29.)
[21] 司新丽. 城市基层图书馆服务可及性:分析框架、调查评价与改进路径[J]. 图书情报工作, 2024, 68(21): 70-78. (SI X L. The service accessibility of urban grass-roots libraries: analytical framework, investigation evaluation and improvement path[J]. Library and information service, 2024, 68(21): 70-78.)
[22] 徐绪堪, 谭丹, 李一铭, 等. 基于SUB-SERVQUAL模型的高校图书馆嵌入式学科服务质量评价研究[J]. 情报理论与实践, 2022, 45(8): 83-89. (XU X K, TAN D, LI Y M, et al. Research on the evaluation of embedded subject service quality of university library based on SUB-SERVQUAL model[J]. Information studies: theory & application, 2022, 45(8): 83-89.)
[23] 王晰巍, 郑国梦, 王铎, 等. 虚拟现实阅读用户交互体验评价指标构建及实证研究[J]. 图书情报工作, 2020, 64(16): 54- 66. (WANG X W, ZHENG G M, WANG D, et al. Evaluation and empirical study of virtual reality interactive reading experience from the perspective of human-computer interaction[J]. Library and information service, 2020, 64(16): 54-66.)
[24] 杨天德, 王芳. 数据摘要质量评价指标体系与撰写模板研制: 以政府开放数据为例[J/OL]. [2025-12-01]. https://link.cnki. net/urlid/11.1541.G2.20250801.1640.002. (YANG T D, WANG F. Developing a quality evaluation index and a writing template for data abstracts: based on open government data[J/OL]. [2025-12- 01]. https://link.cnki.net/urlid/11.1541.G2.20250801.1640.002.)
[25] 尤瑾, 朱学芳. 用户视角下知识类微博信息质量评价模型构建及实证研究[J]. 图书情报工作, 2023, 67(13): 122-132. (YU J, ZHU X F. Model construction and empirical study on information quality evaluation of knowledge-based microblogs from users’ perspective[J]. Library and information service, 2023, 67(13): 122-132.)
[26] 夏立新, 孙丹霞, 王忠义. 网络环境下数字图书馆知识服务用户满意度评价指标体系构建[J]. 图书馆杂志, 2015, 34(3): 27- 34. (XIA L X, SUN D X, WANG Z Y. Evaluation index system of consumer satisfaction for digital library knowledge service in the network environment[J]. Library journal, 2015, 34(3): 27-34.)
[27] 宋雪雁, 张祥青. 基于微信公众号的大学图书馆知识服务质量评价研究[J]. 现代情报, 2020, 40(2): 103-113, 152. (SONG X Y, ZHANG X Q. Research on the evaluation of knowledge service quality of the platform of WeChat of university library[J]. Journal of modern information, 2020, 40(2): 103-113, 152.)
[28] CHANG Y P, WANG X, WANG J D, et al. A survey on evaluation of large language models[J]. ACM transactions on intelligent systems and technology, 2024, 15(3): 1-45.
[29] 中华人民共和国工业和信息化部. “ 机器人+” 应用行动实施方案的通知[EB/OL]. [2025-12-01]. https://www.miit.gov. cn/zwgk/zcjd/art/2023/art_e43b335dd00342b8b980d541806ca edf.html (Ministry of Industry and Information Technology of the People’s Republic of China. Notice on the implementation plan for the “robot+” application action[EB/OL]. [2025-12-01]. https://www.miit.gov.cn/zwgk/zcjd/art/2023/art_e43b335dd00342b8b980d541806caedf.html.)
[30] 禹卫华. 生成式人工智能内容可信度评估指标体系报告(演讲版) [J]. 上海广播电视研究, 2024(3): 38-41. (YU W H. Report on the assessment index system of the credibility of content generated by artificial intelligence (speech version)[J]. Research of radio and television, 2024(3): 38-41.)
[31] 宋雪雁, 管丹丹, 张祥青, 等. 用户满意视角下政务微信知识服务能力评价研究[J]. 情报理论与实践, 2019, 42(3): 120- 126. (SONG X Y, GUAN D D, ZHANG X Q, et al. Evaluation of government WeChat knowledge service capability from the perspective of user satisfaction[J]. Information studies: theory & application, 2019, 42(3): 120-126.)
[32] 中华人民共和国国家互联网信息办公室. 生成式人工智能服务管理暂行办法[EB/OL]. [2025-12-01]. https://www.gov.cn/zhengce/zhengceku/202307/content_6891752.htm. (Cyberspace Administration of China. Provisions on the administration of deep synthesis of internet-based information services. Interim measures for the administration of generative artificial intelligence services[EB/OL]. [2025-12-01]. https://www.gov.cn/zhengce/zhengceku/202307/content_6891752.htm.)
[33] GIBSON D, JACKSON S, SHANMUGASUNDARAM R, et al. Evaluating the efficacy of ChatGPT as a patient education tool in prostate cancer: multimetric assessment[J]. Journal of medical internet research, 2024, 26: e55939.
[34] 全国网络安全标准化技术委员会 . 生成式人工智能服务安全基本要求[EB/OL]. [2025-12-01]. https://dsj. guizhou.gov.cn/xwzx/gnyw/202403/t20240308_86427120. html?isMobile=true (National Technical Committee for Cybersecurity Standardization. Basic security requirements for generative artificial intelligence services[EB/OL]. https://dsj.guizhou.gov.cn/xwzx/gnyw/202403/t20240308_86427120. html?isMobile=true.)
[35] 清华大学人工智能研究院. SuperBench大模型综合能力评测报告 [EB/OL]. [2025-12-01]. https://cloud.tsinghua.edu.cn/f/47717c3407bb4d279e0c/. (Institute of Artificial Intelligence Research, Tsinghua University. SuperBench large model comprehensive capability evaluation report [EB/OL]. [2025-12-01]. https://cloud.tsinghua.edu.cn/f/47717c3407bb4d279e0c/.)
[36] 赵一鸣, 朱奕蓉, 吴林容. 智能语音助手的知识服务能力评价研究[J]. 图书与情报, 2019(4): 132-140. (ZHAO Y M, ZHU Y R, WU L R. Evaluating the knowledge service capability of intelligent voice assistants[J]. Library & information, 2019(4): 132-140.)
[37] The Alan Turing Institute. The rapid rise of generative Al[EB/OL]. [2025-12-01]. https://cetas.turing.ac.uk/publications/rapidrise-generative-ai.
[38] HUANG L Y, ZHANG X, WANG Q, et al. Evaluating media knowledge capabilities of intelligent search dialogue systems: a case study of ChatGPT and New Bing[J]. Journal of the knowledge economy, 2024, 15: 17284-17307.
[39] 中华人民共和国工业和信息化部, 中央网络安全和信息化委员会办公室, 中华人民共和国国家发展和改革委员会, 等. 国家人工智能产业综合标准化体系建设指南[EB/OL]. [2025-12-01]. https://www.gov.cn/zhengce/zhengceku/202407/content_6960720.htm. (Ministry of Industry and Information Technology of The People’s Republic of China, Office of the Central Cyberspace Affairs Commission, National Development and Reform commission, et al. Guidelines for the construction of a comprehensive standardization system for the national artificial intelligence industry [EB/OL]. [2025-12-01]. https://www.gov. cn/zhengce/zhengceku/202407/content_6960720.htm.)
[40] 香港大学经济及工商管理学院深圳研究院. 中文语境下的人工智能大语言模型评测报告[EB/OL]. [2025-12-01]. https://mp.weixin.qq.com/s?__biz=MzU3NjgzNjg4Nw==&mid=2247513000&idx=1&sn=c714152b136135a4244997010333bd1c&ch ksm=fd0f37d0ca78bec6c3690bd1a10d6be78ae6cf0bc0c3b16ff615d70da9ed1b1058ca8940bca9&scene=27. (Shenzhen Research Institute of the University of Hong Kong Faculty of Business and Economics. Evaluation report on large language models of artificial intelligence in Chinese context[EB/OL]. [2025-12-01]. https://mp.weixin.qq.com/s?__biz=MzU3NjgzNjg4Nw==&mid= 2247513000&idx=1&sn=c714152b136135a4244997010333bd1c &chksm=fd0f37d0ca78bec6c3690bd1a10d6be78ae6cf0bc0c3b16ff615d70da9ed1b1058ca8940bca9&scene=27.)
[41] 极客邦科技双数研究院InfoQ研究中心. 2024年大语言模型能力测评报告[EB/OL]. [2025-12-01]. https://www. infoq.cn/minibook/0xhGee6fTUuzDNbj7FdU. (Geekbang Technology Twin Digital Research Institute InfoQ Research Center. 2024 large language model capability evaluation report[EB/OL]. [2025-12-01]. https://www.infoq.cn/minibook/0xhGee6fTUuzDNbj7FdU.)
[42] 张矛, 郑思怡, 宛玲. 高校图书馆数字学术服务能力成熟度评价指标体系构建[J]. 图书情报工作, 2023, 67(14): 12-20. (ZHANG M, ZHENG S Y, WAN L. Construction of evaluation index system of digital scholarship service capability maturity in university libraries[J]. Library and information service, 2023, 67(14): 12-20.)
[43] 中华人民共和国文化和旅游部办公厅. 关于开展第七次全国县级以上公共图书馆评估定级工作的通知[EB/OL]. [2025-12-01]. https://zwgk.mct.gov.cn/zfxxgkml/ggfw/202206/t20220602_933319.html. (Ministry of Culture and Tourism of the People’s Republic of China. Notice on the implementation of the 7th national assessment and grading of public libraries at or above the county level[EB/OL]. [2025-12-01]. https://zwgk.mct. gov.cn/zfxxgkml/ggfw/202206/t20220602_933319.html.)
[44] 关磊. 图书馆数字阅读推广服务效能评价指标体系构建与应用 [J]. 图书馆论坛, 2025, 45(3): 78-88. (GUAN L. Evaluation index system for service effectiveness of library digital reading promotion: construction and application[J]. Library tribune, 2025, 45(3): 78-88.)
[45] SWISHER A R, WU A W, LIU G C, et al. Enhancing health literacy: evaluating the readability of patient handouts revised by ChatGPT’s large language model[J]. Otolaryngology-head and neck surgery, 2024, 171(6): 1751-1757.
[46] 刘健, 王雨菡. 基于CCSI的公共图书馆信息资源服务满意度评价体系构建研究[J]. 情报科学, 2024, 42(4): 155- 166. (LIU J, WANG Y H. The construction of public library information resource service satisfaction evaluation system based on the CCSI model[J]. Information science, 2024, 42(4): 155-166.)
[47] 孙文娉, 陈雅. 融合发展语境下公共图书馆服务效能评价体系研究[J]. 图书馆建设, 2023(6): 155-167. (SUN W P, CHEN Y. Research on service efficiency evaluation system of public library in the context of integrated development[J]. Library development, 2023(6): 155-167.)
[48] 陈思远, 胡凤明, 欧瑞祥. 公共图书馆健康信息服务质量评价研究—— 基于信息生态学理论视角[J]. 图书馆工作与研究, 2024(3): 25-33, 112. (CHEN S Y, HU F M, OU R X. Study on health information service quality evaluation of public libraries: based on the perspective of information ecology theory[J]. Library work and study, 2024(3): 25-33, 112.)
[49] 吴运明, 孔勇, 董同强, 等. 高校图书馆数字学术服务质量评价模型构建及应用研究[J]. 图书馆学研究, 2024(6): 61- 69. (WU Y M, KONG Y, DONG T Q, et al. Research on the construction and application of digital academic service quality evaluation model in university libraries[J]. Library science research, 2024(6): 61-69.)
[50] 林崇青. 基于扎根理论的公共图书馆数字文化服务绩效评价体系构建与实证研究[J]. 图书馆理论与实践, 2024(3): 26-34. (LIN C Q. Construction and empirical research on the performance evaluation system of public library digital cultural service based on grounded theory[J]. Library theory and practice, 2024(3): 26-34.)
[51] 孔泳欣, 彭国超, 李颖. 知识链视角下数字图书馆知识服务能力评价研究[J]. 情报理论与实践, 2021, 44(11): 73-79. (KONG Y X, PENG G C, LI Y. An evaluation system of knowledge service capability in digital library based on knowledge chain theory[J]. Information studies: theory & application, 2021, 44(11): 73-79.)
[52] SAATY T L. Multicriteria decision making: the analytic hierarchy process[M]. Pittsburgh: RWS Publications, 1988.
[53] 朱喜安, 魏国栋. 熵值法中无量纲化方法优良标准的探讨[J]. 统计与决策, 2015(2): 12-15. (ZHU X A, WEI G D. Discussion on the excellent standards of dimensionless methods in the entropy method [J]. Statistics & decision, 2015(2): 12-15.)
[54] 虞晓芬, 傅玳. 多指标综合评价方法综述[J]. 统计与决策, 2004(11): 119-121. (YU X F, FU D. A review of multi - indicator comprehensive evaluation methods[J]. Statistics & decision, 2004(11): 119-121.)
[55] MERAL G, ATES S, GÜNAY S, et al. Comparative analysis of ChatGPT, Gemini and emergency medicine specialist in ESI triage assessment[J]. American journal of emergency medicine, 2024, 81: 146-150.
[56] 王芳, 魏中瀚, 连芷萱. 政务智能问答系统评价指标体系构建与测评问题编制[J]. 图书情报知识, 2023, 40(6): 98-111. (WANG F, WEI Z H, LIAN Z X. Developing an evaluation index system and its testing questions for government intelligent Q&A systems[J]. Documentation, information & knowledge, 2023, 40(6): 98-111.)
[57] 宋士杰, 赵宇翔, 朱庆华. 从ELIZA到ChatGPT:人智交互体验中的AI生成内容( AIGC)可信度评价[J]. 情报资料工作, 2023, 44(4): 35-42. (SONG S J, ZHAO Y X, ZHU Q H. From ELIZA to ChatGPT: AI-generated content(AIGC) credibility evaluation in human-intelligent interactive experience[J]. Information and documentation services, 2023, 44(4): 35-42.)
[58] HUANG D, CHUEH H. Chatbot usage intention analysis: veterinary consultation[J]. Journal of innovation & knowledge, 2021, 6(3): 135-144.
[59] DE OLIVEIRA G G, LIZARELLI F L, TEIXEIRA J G, et al. Curb your enthusiasm: examining the customer experience with Alexa and its marketing outcomes[J]. Journal of retailing and consumer services, 2023, 71: 103220.
文章导航

/

〈 〉