图书情报工作

图书情报工作 >

2025 , Vol. 69 >Issue 24: 81 - 93

DOI: https://doi.org/10.13266/j.issn.0252-3116.2025.24.006

研究论文

基于多维度特征识别的算法实体学术影响力评价

  • 王玉琢 ,
  • 章成志 ,
  • 张恒 ,
  • 赵毅
展开
  • 1 安徽大学管理学院 合肥 230601;
    2 南京理工大学经济管理学院 南京 210094;
    3 华中师范大学信息管理学院 武汉 430079
王玉琢，讲师，博士;章成志，教授，博士，博士生导师，通信作者，E-mail:zhangcz@njust.edu.cn;张恒，讲师，博士;赵毅，讲师，博士。

收稿日期: 2024-12-24

  修回日期: 2025-02-19

  网络出版日期: 2025-12-05

基金资助

本文系国家社会科学基金青年项目“基于学术全文本的算法实体语义关系识别与演化路径研究”（项目编号：24CTQ027）研究成果之一。

收起

Academic Influence Evaluation for Algorithm Entities Based on the Multidimensional Feature Recognition

  • Wang Yuzhuo ,
  • Zhang Chengzhi ,
  • Zhang Heng ,
  • Zhao Yi
Expand
  • 1 School of Management, Anhui University, Hefei 230601;
    2 School of Economics and Management, Nanjing University of Science and Technology, Nanjing 210094;
    3 School of Information Management, Central China Normal University, Wuhan 430079

Received date: 2024-12-24

  Revised date: 2025-02-19

  Online published: 2025-12-05

Supported by

This work is supported by the youth project of the National Social Science Fund of China, titled “Identifying the Semantic Relationship and Evolutionary Paths of Algorithm Entities Based on the Full-text of Academic Papers” (Grant No. 24CTQ027).

Fold

摘要

[目的/意义]算法是人工智能时代的核心构成要素之一，也是现代科学研究的核心研究对象和研究方法。对特定领域的算法学术影响力进行量化评估，可为科学家选择、改进和应用算法提供参考。[方法/过程]基于自然语言处理领域会议论文全文，开展算法实体及其被提及频次、位置、动机等多维度特征的自动识别，在此基础上构建融合不同内容特征的影响力指标，对算法实体在NLP领域的学术影响力进行分析。[结果/结论]算法实体在论文“方法”相关章节、以“直接使用”的动机被提及最多。相较于单一特征指标，综合频次、位置、动机的综合指标所得结果最接近专家评价的金标准结果。在不同算法实体中，里程碑式的经典算法、性能优越的新兴算法可获得更高的学术影响力。结果为特定类型的学术实体影响力评价提供完整的实体获取、特征分析、影响力量化研究框架，为基于细粒度实体的知识服务奠定基础。

关键词： 算法实体; 学术全文本; 影响力评估; 内容特征识别

本文引用格式

王玉琢 , 章成志 , 张恒 , 赵毅 . 基于多维度特征识别的算法实体学术影响力评价[J]. 图书情报工作, 2025 , 69(24) : 81 -93 . DOI: 10.13266/j.issn.0252-3116.2025.24.006

Abstract

[Purpose/Significance] Algorithms are one of the fundamental components of the artificial intelligence (AI) technology and also the core research object and method in modern scientific research. Quantitative assessment of algorithms’ academic influence within specific fields can help scientists select, refine, and apply algorithms. [Method/Process] Based on conference papers in the field of natural language processing (NLP), this study automatically identified the algorithm entities and frequency, locations, motivations of its mention, constructed influence indicators integrating different content features, and analyzed its academic influence in the NLP field. [Result/Conclusion] Algorithm entities are most frequently mentioned in the “Methods” section of academic papers, motivated by “direct use.” Compared to single-feature indicators, the metric combining frequency, location, and motivation yields results closest to the expert-evaluated gold standard. Among different algorithm entities, classic and landmark algorithms with superior performance have achieved higher academic influence. The results provide a complete research framework of entity acquisition, characterization, and impact quantification for evaluating the impact of specific types of academic entities, which lays the foundation for knowledge services based on fine-grained entities.

Key words： algorithm entities; full-text content of academic paper; influence evaluation; context features recognition

参考文献

[1] 李金磊. （两会观察）从“互联网+”到“人工智能+”，中国以“智”提“质”[EB/OL]. [2025-10-25]. https://www.chinanews.com.cn/cj/2024/03-07/10176030.shtml. (LI J L. From “Internet+” to “Artificial Intelligence+”, China improves “quality” with “wisdom” (Observations on the two sessions) [EB/OL]. [2025-10-25]. https://www.chinanews.com.c n/cj/2024/03-07/10176030.shtml.)
[2] 李猛. “人工智能+”赋能新质生产力发展——内在机理与路径探索[J]. 北京航空航天大学学报(社会科学版), 2024, 37(4): 127-137. (LI M. “Artificial Intelligence +” empowers the development of new quality productive forces: exploring the intrinsic mechanism and paths[J]. Journal of Beijing University of Aeronautics and Astronautics (social sciences edition), 2024, 37(4): 127-137.)
[3] 杜传忠, 疏爽, 李泽浩. 新质生产力促进经济高质量发展的机制分析与实现路径[J]. 经济纵横, 2023(12): 20-28. (DU C Z, YU S, LI Z H. Mechanism and path of new quality productivity in promoting high-quality economic development[J]. Economic review journal, 2023(12): 20-28.)
[4] WU X, KUMAR V, QUINLAN J R, et al. Top 10 algorithms in data mining[J]. Knowledge and information systems, 2018, 14(1): 1-37.
[5] TAHAMTAN I, BORNMANN L. What do citation counts measure? An updated review of studies on citations in scientific documents published between 2006 and 2018[J]. Scientometrics, 2019, 121(3): 1635-1684.
[6] HERNÁNDEZ-ALVAREZ M, SORIANO J M G, MARTÍNEZ-BARCO P. Citation function, polarity and influence classification[J]. Natural language engineering, 2017, 23(4): 561-588.
[7] 卢超, 章成志, 王玉琢, 等. 语义特征分析的深化——学术文献的全文计量分析研究综述[J]. 中国图书馆学报, 2021, 47(2): 110-131. (LU C, ZHANG C Z, WANG Y Z, et al. Strengthened analyses of semantic features: review of full-text bibliometrics of academic documents [J]. Journal of library science in China, 2021, 47(2): 110-131.)
[8] WANG Y, ZHANG C. Using the full-text content of academic articles to identify and evaluate algorithm entities in the domain of natural language processing[J]. Journal of informetrics, 2020, 14(4): 101091.
[9] BHATIA S, TUAROB S, MITRA P, et al. An algorithm search engine for software developers[C]// Proceedings of the 3rd international workshop on search-driven development: users, infrastructure, tools, and evaluation. New York: ACM, 2011: 13-16.
[10] WANG Y, ZHANG C. Using full-text of research articles to analyze academic impact of algorithms[C]// CHOWDHURY G, MCLEOD J, GILLET V, WILLETT P. Transforming digital worlds. iConference 2018. Sheffield: Springer, 2018: 395-401.
[11] WANG Y, ZHANG C. Finding more methodological entities from academic articles via iterative strategy: a preliminary study[C]// Proceedings of the 17th international conference on scientometrics and informetrics. Rome: ISSI Society, 2019: 2702–2703.
[12] TUAROB S, MITRA P, Giles C L. A classification scheme for algorithm citation function in scholarly works[C]// Proceedings of the 13th ACM/IEEE-CS joint conference on digital libraries. New York: ACM, 2013: 367-368.
[13] SAFDER I, HASSAN S U, VISVIZI A, et al. Deep learning-based extraction of algorithmic metadata in full-text scholarly documents[J]. Information processing & management, 2020, 57(6): 102269.
[14] 丁睿祎, 王玉琢, 章成志. 基于学术论文全文内容的特定领域算法实体抽取研究[J]. 数字图书馆论坛, 2022(3): 2-14. (DING R Y, WANG Y Z, ZHANG C Z. Extraction algorithmic entity from full text of academic articles in special domain[J]. Digital library forum, 2022(3): 2-14.)
[15] TUAROB S, KANG S, WETTAYAKORN P, et al. Automatic classification of algorithm citation functions in scientific literature[J]. IEEE transactions on knowledge and data engineering, 2019, 32(10): 1881-1896.
[16] WANG Y, ZHANG C, SONG M, et al. Exploring academic influence of algorithms by co-occurrence network based on full-text of academic papers[J]. Aslib journal of information management, 2025, 77(4): 651-680.
[17] ZHA H, CHEN W, LI K, et al. Mining algorithm roadmap in scientific publications[C]// Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining. Anchorage: ACM, 2019: 1083-1092.
[18] Stack Overflow. Stack overflow developer survey 2024[EB/OL]. [2025-10-25]. https://survey.stackoverflow.co/2024/.
[19] GARFIELD E. Citation indexing - its theory and application in science, technology, and humanities[M]. New York: Wiley, 1979.
[20] DING R, WANG Y, ZHANG C. Investigating citation of algorithm in full-text of academic articles in nlp domain: a preliminary study[C]// Proceedings of the 17th international conference on scientometrics and informetrics. Rome: ISSI Society, 2019: 2726-2727.
[21] HOWISON J, DEELMAN E, MCLENNAN M J, et al. Understanding the scientific software ecosystem and its impact: current and future measures[J]. Research evaluation, 2015, 24(4): 454-470.
[22] LI K, CHEN P Y, YAN E. Challenges of measuring software impact through citations: an examination of the lme4 R package[J]. Journal of informetrics, 2019, 13(1): 449-461.
[23] PAN X, YAN E, WANG Q, et al. Assessing the impact of software on science: a bootstrapped learning of software entities in full-text papers[J]. Journal of informetrics, 2015, 9(4): 860-871.
[24] PAN X, YAN E, CUI M, et al. How important is software to library and information science research? A content analysis of full-text publications[J]. Journal of informetrics, 2019, 13(1): 397-406.
[25] ZHAO R, WEI M. Impact evaluation of open source software: an Altmetrics perspective[J]. Scientometrics, 2017, 110(2): 1017-1033.
[26] MA B, ZHANG C, WANG Y, et al. Enhancing identification of structure function of academic articles using contextual information[J]. Scientometrics, 2022, 127(2): 885-925.
[27] WANG Y, XIANG Y, ZHANG C. Exploring motivations for algorithm mention in the domain of natural language processing: a deep learning approach[J]. Journal of informetrics, 2024, 18(4): 101550.
[28] MITKOV R. The Oxford handbook of computational linguistics (Oxford handbooks) [M]. New York: Oxford University Press, 2005.
[29] MCHUGH M L. Interrater reliability: the kappa statistic[J]. Biochemia medica, 2012, 22(3): 276-282.
[30] VOOS H, DAGAEV K S. Are all citations equal? Or, did we op. cit. your idem?[J]. Journal of academic librarianship, 1976, 1(6): 19-21.
[31] HERLACH G. Can retrieval of information from citation indexes be simplified? multiple mention of a reference as a characteristic of the link between cited and citing article[J]. Journal of the American Society for Information Science, 1978, 29(6): 308-310.
[32] 夏红玉, 胡潜, 王忠义. 基于引文重要性的知识流动主路径分析[J]. 情报学报, 2022, 41(5): 451-462. (XIA H Y, HU Q, WANG Z Y. Tracing the knowledge flow main path based on important citations[J]. Journal of the China Society for Scientific and Technical Information, 2022, 41(5): 451-462.)
[33] MARICIC S, SPAVENTI J, PAVICIC L, et al. Citation context versus the frequency counts of citation histories[J]. Journal of the American Society for Information Science & technology, 1998, 49(6): 530-540.
[34] 李铮, 邓三鸿, 孔嘉, 等. 学者学术影响力识别研究——基于引文全数据的视角[J]. 图书情报工作, 2020, 64(12): 87-94. (LI Z, DENG S H, KONG J, et al. Research on scholar’s academic influence: from the perspective of full citation data [J]. Library and information service, 2020, 64(12): 87-94.)
[35] 周清清. 基于多源异构数据的图书影响力评价研究[D]. 南京: 南京理工大学, 2018. (ZHOU Q Q. Book impact assessment based on multi-source heterogeneous data[D]. Nanjing: Nanjing University of Science And Technology, 2018.)
[36] ZHANG C, ZHOU Q. Assessing books’ depth and breadth via multi-level mining on tables of contents[J]. Journal of informetrics, 2020, 14(2): 101032.
[37] 张光耀, 邵文君, 谢维熙, 等. 同行评议有效吗——基于有效市场假说的探究[J]. 图书情报知识, 2023, 40(2): 105-116. (ZHANG G Y, SHAO W J, XIE W X, et al. Is peer review effective? An exploration based on efficient market hypothesis [J]. Documentation, information & knowledge, 2023, 40(2): 105-116.)
[38] 张旭, 吕冬晴, 阮选敏, 等. 融合文献计量与同行评议共识的期刊评价方法研究[J]. 情报学报, 2022, 41(5): 486-496. (ZHANG X, LV D Q, RUAN X M, et al. Journal evaluation method based on the integration of peer-review consensus and bibliometric indicators [J]. Journal of the China Society for Scientific and Technical Information, 2022, 41(5): 486-496.)
[39] HOCHREITER S, SCHMIDHUBER J. Long Short-Term Memory[J]. Neural computation, 1997, 9(8): 1735-1780.
[40] LIN J. Divergence measures based on the Shannon entropy[J]. IEEE transactions on information theory, 1991, 37(1): 145-151.
Options
文章导航

/

〈 〉