情报研究

AIGC质量评估指标体系构建研究——以ChatGPT为例

  • 郭亚军 ,
  • 刘振阳 ,
  • 冯思倩 ,
  • 袁一鸣
展开
  • 1 郑州航空工业管理学院信息管理学院 郑州 450046;
    2 武汉大学信息管理学院 武汉 430072
郭亚军,教授,博士,硕士生导师,E-mail:guoyajun0619@126.com;刘振阳,硕士研究生;冯思倩,硕士研究生;袁一鸣,博士研究生。

收稿日期: 2024-04-29

  修回日期: 2024-07-24

  网络出版日期: 2024-11-25

基金资助

本文系国家社会科学基金项目“元宇宙场域下智慧图书馆空间服务创新研究”(项目编号:23BTQ011)研究成果之一。

Research on the Construction of Quality Evaluation Index System of AIGC——Taking ChatGPT as an Example

  • Guo Yajun ,
  • Liu Zhenyang ,
  • Feng Siqian ,
  • Yuan Yiming
Expand
  • 1 School of Information Management, Zhengzhou University of Aeronautics, Zhengzhou 450046;
    2 School of Information Management, Wuhan University, Wuhan 430072

Received date: 2024-04-29

  Revised date: 2024-07-24

  Online published: 2024-11-25

Supported by

This work is supported by the National Social Science Fund of China Project titled “Research on Innovative Spatial Services of Smart Libraries in the Metaverse Field”(Grant No.23BTQ011).

摘要

[目的/意义] 构建AIGC质量评估指标体系,有助于用户识别高质量的AIGC以及AIGC服务商提高内容质量。[方法/过程] 通过文献检索和用户访谈获得原始数据,使用扎根理论研究方法对原始数据进行三级编码,提取AIGC质量评估指标。在此基础上,使用问卷调查法和层次分析法确定指标权重,构建AIGC质量评估指标体系。[结果/结论] AIGC质量评估指标体系包括内容特征、表达特征、效用特征、技术特征4个一级指标和个性化、高效性、易理解性、可解释性等21个二级指标,其中内容特征的权重最大,对于AIGC质量的影响程度最高。此外,研究发现AIGC与PGC、UGC在质量评估方面存在一定的评估异同,可追溯性、类人性、可塑性、可解释性等是AIGC质量评估特有的指标。

本文引用格式

郭亚军 , 刘振阳 , 冯思倩 , 袁一鸣 . AIGC质量评估指标体系构建研究——以ChatGPT为例[J]. 图书情报工作, 2024 , 68(22) : 93 -104 . DOI: 10.13266/j.issn.0252-3116.2024.22.009

Abstract

[Purpose/Significance] The construction of quality evaluation index system of AIGC is helpful for users to identify high-quality AIGC and AIGC service providers to improve content quality. [Method/Process] The original data were obtained through literature retrieval and user interviews, and were encoded at three levels based on grounded theory to extract the quality evaluation indexes of AIGC. On this basis, a questionnaire survey and analytic hierarchy process were used to determine the weight of each index, and the quality evaluation index system of AIGC was constructed. [Result/Conclusion] The quality evaluation index system of AIGC includes 4 first-level indexes such as content characteristics, expression characteristics, utility characteristics and technology characteristics, along with 21 second-level indexes such as personalization, efficiency, intelligibility and interpretability. Among these, content characteristics have the highest weight and the greatest impact on AIGC quality. In addition, the study finds that AIGC, PGC and UGC have some similarities and differences in the evaluation of information quality. Traceability, humanism, plasticity and interpretability are unique indexes of AIGC quality evaluation.

参考文献

[1] OPENAI.Creating video from text[EB/OL].[2024-09-25].https://openai.com/sora?ref=aihub.cn.
[2] LIU Y, ZHANG K, LI Y, et al.Sora: a review on background, technology, limitations, and opportunities of large vision models[J].arXiv preprint arXiv:2402.17177, 2024.
[3] ROMERO A.OpenAI Sora: one step away from the matrix[EB/OL].[2024-09-25].https://www.thealgorithmicbridge.com/p/openai-sora-one-step-away-from-the.
[4] 中国信息通信研究院, 京东探索研究院.人工智能生成内容(AIGC)白皮书[R/OL].[2024-07-20].http://www.caict.ac.cn/kxyj/qwfb/bps/202209/P020220902534520798735.pdf.(China Academy of Information and Communication, Jingdong Exploration Research Institute.AI Generated Content (AIGC) White Paper[R/OL].[2024-07-20].http://www.caict.ac.cn/kxyj/qwfb/bps/202209/P020220902534520798735.pdf.)
[5] 朱光辉, 王喜文.ChatGPT的运行模式、关键技术及未来图景[J].新疆师范大学学报(哲学社会科学版), 2023, 44(4): 113-122.(ZHU G H, WANG X W.ChatGPT: operation mode, key technology and future prospects[J].Journal of Xinjiang Normal University (edition of philosophy and social sciences), 2023, 44(4): 113-122.)
[6] 王静静, 叶鹰.生成式AI及其GPT类技术应用对信息管理与传播的变革探析[J].中国图书馆学报, 2023, 49(6): 41-50.(WANG J J, YE Y.A probe into the Generative AI and GPT-type technical applications with transform for information management and communication[J].Journal of library science in China, 2023, 49(6): 41-50.)
[7] 刘逸伦, 黄微, 张晓君, 等.AIGC赋能的科技情报智能服务:特征、场景与框架[J].现代情报, 2023, 43(12): 88-99.(LIU Y L, HUANG W, ZHANG X J, et al.Research on the Artificial Intelligence-Generated Content enabled intelligent scientific and technical information service framework[J].Journal of modern information, 2023, 43(12): 88-89.)
[8] 赵杨, 张雪, 范圣悦.AIGC驱动的智慧图书馆转型:框架、路径与挑战[J].情报理论与实践, 2023, 46(7): 9-16.(ZHAO Y, ZHANG X, FAN S Y.AIGC-driven intelligent library transformation: framework, pathways and challenges[J].Information studies: theory & application, 2023, 46(7): 9-16.)
[9] 吴若航, 茆意宏.ChatGPT热潮下的图书馆服务:理念、机遇与破局[J].图书与情报, 2023(2): 34-41.(WU R H, MAO Y H.Library services under the ChatGPT boom: concepts, opportunities, and breakthroughs[J].Library & information, 2023(2): 34-41.)
[10] 王鹏涛, 徐润婕.AIGC介入知识生产下学术出版信任机制的重构研究[J].图书情报知识, 2023, 40(5): 87-96.(WANG P T, XU R J.Reconstruction of trust mechanism in academic publishing under the intervention of AIGC in knowledge production[J].Documentation, information & knowledge, 2023, 40(5): 87-96.)
[11] 李白杨, 白云, 詹希旎, 等.人工智能生成内容(AIGC)的技术特征与形态演进[J].图书情报知识, 2023, 40(1): 66-74.(LI B Y, BAI Y, ZHAN X N, et al.The technical features and aromorphosis of artificial intelligence generated content (AIGC)[J].Documentation, information & knowledge, 2023, 40(1): 66-74.)
[12] 郭亚军, 马慧芳, 张鑫迪, 等.ChatGPT赋能图书馆知识服务:原理、场景与进路[J].图书馆建设, 2024(3): 60-68.(GUO Y J, MA H F, ZHANG X D, et al.ChatGPT empowers library knowledge services: principles, scenarios, and approaches[J].Library development, 2024(3): 60-68.)
[13] BAIDOO-ANU D, ANSAH L O.Education in the era of generative artificial intelligence (AI): understanding the potential benefits of ChatGPT in promoting teaching and learning[J].Journal of AI, 2023, 7(1): 52-62.
[14] MESKÓ B, TOPOL E J.The imperative for regulatory oversight of large language models (or generative AI) in healthcare[J].NPJ digital medicine, 2023, 6(1): 120.
[15] LIU Y, YANG Z, YU Z, et al.Generative artificial intelligence and its applications in materials science: current situation and future perspectives[J].Journal of materiomics, 2023, 9(4): 798-816.
[16] WAND Y, WANG R Y.Anchoring data quality dimensions in ontological foundations[J].Communications of the ACM, 1996, 39(11): 86-95.
[17] 李蕾, 张琳琳, 王傲, 等.社交媒体环境下学术型用户生成内容质量评估研究[J].情报理论与实践, 2023, 46(2): 175-183.(LI L, ZHANG L L, WANG A, et al.Quality evaluation of academic user generated content on social media[J].Information studies: theory & application, 2023, 46(2): 175-183.)
[18] EPPLER M J, WITTIG D.Conceptualizing information quality: a review of information quality frameworks from the last ten years[C]//Proceedings of the Conference on Information Quality.Cambridge: Massachusetts Institute of Technology, 2000: 83-96.
[19] LEE Y W, STRONG D M, KAHN B K, et al.AIMQ: a methodology for information quality assessment[J].Information & management, 2002, 40(2): 133-146.
[20] 朱益平, 杜海娇, 张佳, 等.基于RS-BP神经网络的政务微信公众号信息质量评价模型研究[J].情报科学, 2021, 39(2): 54-61, 69.(ZHU Y P, DU H J, ZHANG J, et al.Information quality evaluation model research of government WeChat public account based on RS-BP neural networks[J].Information science, 2021, 39(2): 54-61, 69.)
[21] 李全喜, 徐嘉徽, 魏骏巍, 等.基于BP神经网络的共享服务平台资源信息质量评价研究——以短租类共享服务平台为例[J].图书情报工作, 2019, 63(10): 125-133.(LI Q X, XU J H, WEI J W, et al.Resource information quality evaluation of shared service platform based on BP neural network: taking short-term rental shared service platform as an example[J].Library and information service, 2019, 63(10): 125-133.)
[22] 徐文强, 刘春年, 周涛.大数据环境下应急信息质量评估体系研究[J].图书情报工作, 2020, 64(2): 50-58.(XU W Q, LIU C N, ZHOU T.Research on emergency information quality evaluation system in big data environment[J].Library and information service, 2020, 64(2): 50-58.)
[23] TAO D, LEROUGE C, SMITH K J, et al.Defining information quality into health websites: a conceptual framework of health website information quality for educated young adults[J].JMIR human factors, 2017, 4(4): e6455.
[24] CHUENCHOM S.User-centered evaluation of the quality of blogs[D].Denton: University of North Texas, 2011.
[25] GLASER B G, STRAUSS A L, STRUTZEL E.The discovery of grounded theory; strategies for qualitative research[J].Nursing research, 1968, 17(4): 364.
[26] 夏昊翔, 王众托, 党延忠.关于信息系统概念基础的一点思考[J].系统工程理论与实践, 2001(10): 17-24.(XIA H X, WANG Z T, DANG Y Z.On the conceptual foundation of information systems[J].Systems engineering-theory & practice, 2001(10): 17-24.)
[27] HOLSTI O R.Content analysis for the social sciences and humanities[M].California: Addison-Westly Publishing Company, 1969.
[28] 李贺, 张世颖.移动互联网用户生成内容质量评价体系研究[J].情报理论与实践, 2015, 38(10): 6-11, 37.(LI H, ZHANG S Y.Research on quality evaluation system of mobile Internet user generated content[J].Information studies: theory & application, 2015, 38(10): 6-11, 37.)
[29] 万安伦, 张小凡, 曹培培.ChatGPT浪潮下的数字出版:模式创新与行业挑战[J].中国编辑, 2023(10): 14-20.(WAN A L, ZHANG X F, CAO P P.Digital publishing in the ChatGPT wave: Model innovation and industry challenges[J].Chinese editors journal, 2023(10): 14-20.)
[30] ALASADI E A, BAIZ C R.Generative AI in education and research: Opportunities, concerns, and solutions[J].Journal of chemical education, 2023, 100(8): 2965-2971.
[31] 储节旺, 杜秀秀, 李佳轩.人工智能生成内容对智慧图书馆服务的冲击及应用展望[J].情报理论与实践, 2023, 46(5): 6-13.(CHU J W, DU X X, LI J X.The impact and application prospects of artificial intelligence generated content on smart library services[J].Information studies: theory & application, 2023, 46(5): 6-13.)
[32] 詹希旎, 李白杨, 孙建军.数智融合环境下AIGC的场景化应用与发展机遇[J].图书情报知识, 2023, 40(1): 75-85, 55.(ZHAN X N, LI B Y, SUN J J.Application scenarios and development opportunities of AIGC in the digital intelligence integration environment[J].Documentation, information & knowledge, 2023, 40(1): 75-85, 55.)
[33] 夏德元.AIGC时代的知识生产逻辑与出版流程再造[J].中国编辑, 2023(9): 46-50.(XIA D Y.Reengineering of knowledge production logic and publishing process in AIGC era[J].Chinese editors journal, 2023(9): 46-50.)
[34] 曹树金, 曹茹烨.从ChatGPT看生成式AI对情报学研究与实践的影响[J].现代情报, 2023, 43(4): 3-10.(CAO S J, CAO R Y.Influence of Generative AI on the research and practice of Information Science from the perspective of ChatGPT[J].Journal of modern information, 2023, 43(4): 3-10.)
[35] 吴丹, 孙国烨.迈向可解释的交互式人工智能:动因、途径及研究趋势[J].武汉大学学报(哲学社会科学版), 2021, 74(5): 16-28.(WU D, SUN G Y.Towards explainable interactive artificial intelligence: motivations, approaches, and research trends[J].Wuhan University journal (philosophy & social science), 2021, 74(5): 16-28.)
[36] 段尧清, 朱永迪, 蔡启宾, 等.突发公共卫生事件下公众持续使用政府信息行为影响因素研究——基于QCA的触发路径及演化分析[J].信息资源管理学报, 2022, 12(3): 89-99.(DUAN Y Q, ZHU Y D, CAI Q B, et al.Research on influencing factors of government information public sustainable use behavior under public health emergency: trigger path and evolution analysis based on QCA[J].Journal of information resources management, 2022, 12(3): 89-99.)
文章导航

/

〈 〉