Library and Information Service

Library and Information Service >

2025 , Vol. 69 >Issue 24: 81 - 93

DOI: https://doi.org/10.13266/j.issn.0252-3116.2025.24.006

SPECIAL TOPIC: Scientific Innovation and Knowledge Transformation from the Perspective of New Quality Productive Forces

Academic Influence Evaluation for Algorithm Entities Based on the Multidimensional Feature Recognition

  • Wang Yuzhuo ,
  • Zhang Chengzhi ,
  • Zhang Heng ,
  • Zhao Yi
Expand
  • 1 School of Management, Anhui University, Hefei 230601;
    2 School of Economics and Management, Nanjing University of Science and Technology, Nanjing 210094;
    3 School of Information Management, Central China Normal University, Wuhan 430079

Received date: 2024-12-24

  Revised date: 2025-02-19

  Online published: 2025-12-05

Supported by

This work is supported by the youth project of the National Social Science Fund of China, titled “Identifying the Semantic Relationship and Evolutionary Paths of Algorithm Entities Based on the Full-text of Academic Papers” (Grant No. 24CTQ027).

Fold

Abstract

[Purpose/Significance] Algorithms are one of the fundamental components of the artificial intelligence (AI) technology and also the core research object and method in modern scientific research. Quantitative assessment of algorithms’ academic influence within specific fields can help scientists select, refine, and apply algorithms. [Method/Process] Based on conference papers in the field of natural language processing (NLP), this study automatically identified the algorithm entities and frequency, locations, motivations of its mention, constructed influence indicators integrating different content features, and analyzed its academic influence in the NLP field. [Result/Conclusion] Algorithm entities are most frequently mentioned in the “Methods” section of academic papers, motivated by “direct use.” Compared to single-feature indicators, the metric combining frequency, location, and motivation yields results closest to the expert-evaluated gold standard. Among different algorithm entities, classic and landmark algorithms with superior performance have achieved higher academic influence. The results provide a complete research framework of entity acquisition, characterization, and impact quantification for evaluating the impact of specific types of academic entities, which lays the foundation for knowledge services based on fine-grained entities.

Key words： algorithm entities; full-text content of academic paper; influence evaluation; context features recognition

Cite this article

Wang Yuzhuo , Zhang Chengzhi , Zhang Heng , Zhao Yi . Academic Influence Evaluation for Algorithm Entities Based on the Multidimensional Feature Recognition[J]. Library and Information Service, 2025 , 69(24) : 81 -93 . DOI: 10.13266/j.issn.0252-3116.2025.24.006

References

[1] 李金磊. （两会观察）从“互联网+”到“人工智能+”，中国以“智”提“质”[EB/OL]. [2025-10-25]. https://www.chinanews.com.cn/cj/2024/03-07/10176030.shtml. (LI J L. From “Internet+” to “Artificial Intelligence+”, China improves “quality” with “wisdom” (Observations on the two sessions) [EB/OL]. [2025-10-25]. https://www.chinanews.com.c n/cj/2024/03-07/10176030.shtml.)
[2] 李猛. “人工智能+”赋能新质生产力发展——内在机理与路径探索[J]. 北京航空航天大学学报(社会科学版), 2024, 37(4): 127-137. (LI M. “Artificial Intelligence +” empowers the development of new quality productive forces: exploring the intrinsic mechanism and paths[J]. Journal of Beijing University of Aeronautics and Astronautics (social sciences edition), 2024, 37(4): 127-137.)
[3] 杜传忠, 疏爽, 李泽浩. 新质生产力促进经济高质量发展的机制分析与实现路径[J]. 经济纵横, 2023(12): 20-28. (DU C Z, YU S, LI Z H. Mechanism and path of new quality productivity in promoting high-quality economic development[J]. Economic review journal, 2023(12): 20-28.)
[4] WU X, KUMAR V, QUINLAN J R, et al. Top 10 algorithms in data mining[J]. Knowledge and information systems, 2018, 14(1): 1-37.
[5] TAHAMTAN I, BORNMANN L. What do citation counts measure? An updated review of studies on citations in scientific documents published between 2006 and 2018[J]. Scientometrics, 2019, 121(3): 1635-1684.
[6] HERNÁNDEZ-ALVAREZ M, SORIANO J M G, MARTÍNEZ-BARCO P. Citation function, polarity and influence classification[J]. Natural language engineering, 2017, 23(4): 561-588.
[7] 卢超, 章成志, 王玉琢, 等. 语义特征分析的深化——学术文献的全文计量分析研究综述[J]. 中国图书馆学报, 2021, 47(2): 110-131. (LU C, ZHANG C Z, WANG Y Z, et al. Strengthened analyses of semantic features: review of full-text bibliometrics of academic documents [J]. Journal of library science in China, 2021, 47(2): 110-131.)
[8] WANG Y, ZHANG C. Using the full-text content of academic articles to identify and evaluate algorithm entities in the domain of natural language processing[J]. Journal of informetrics, 2020, 14(4): 101091.
[9] BHATIA S, TUAROB S, MITRA P, et al. An algorithm search engine for software developers[C]// Proceedings of the 3rd international workshop on search-driven development: users, infrastructure, tools, and evaluation. New York: ACM, 2011: 13-16.
[10] WANG Y, ZHANG C. Using full-text of research articles to analyze academic impact of algorithms[C]// CHOWDHURY G, MCLEOD J, GILLET V, WILLETT P. Transforming digital worlds. iConference 2018. Sheffield: Springer, 2018: 395-401.
[11] WANG Y, ZHANG C. Finding more methodological entities from academic articles via iterative strategy: a preliminary study[C]// Proceedings of the 17th international conference on scientometrics and informetrics. Rome: ISSI Society, 2019: 2702–2703.
[12] TUAROB S, MITRA P, Giles C L. A classification scheme for algorithm citation function in scholarly works[C]// Proceedings of the 13th ACM/IEEE-CS joint conference on digital libraries. New York: ACM, 2013: 367-368.
[13] SAFDER I, HASSAN S U, VISVIZI A, et al. Deep learning-based extraction of algorithmic metadata in full-text scholarly documents[J]. Information processing & management, 2020, 57(6): 102269.
[14] 丁睿祎, 王玉琢, 章成志. 基于学术论文全文内容的特定领域算法实体抽取研究[J]. 数字图书馆论坛, 2022(3): 2-14. (DING R Y, WANG Y Z, ZHANG C Z. Extraction algorithmic entity from full text of academic articles in special domain[J]. Digital library forum, 2022(3): 2-14.)
[15] TUAROB S, KANG S, WETTAYAKORN P, et al. Automatic classification of algorithm citation functions in scientific literature[J]. IEEE transactions on knowledge and data engineering, 2019, 32(10): 1881-1896.
[16] WANG Y, ZHANG C, SONG M, et al. Exploring academic influence of algorithms by co-occurrence network based on full-text of academic papers[J]. Aslib journal of information management, 2025, 77(4): 651-680.
[17] ZHA H, CHEN W, LI K, et al. Mining algorithm roadmap in scientific publications[C]// Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining. Anchorage: ACM, 2019: 1083-1092.
[18] Stack Overflow. Stack overflow developer survey 2024[EB/OL]. [2025-10-25]. https://survey.stackoverflow.co/2024/.
[19] GARFIELD E. Citation indexing - its theory and application in science, technology, and humanities[M]. New York: Wiley, 1979.
[20] DING R, WANG Y, ZHANG C. Investigating citation of algorithm in full-text of academic articles in nlp domain: a preliminary study[C]// Proceedings of the 17th international conference on scientometrics and informetrics. Rome: ISSI Society, 2019: 2726-2727.
[21] HOWISON J, DEELMAN E, MCLENNAN M J, et al. Understanding the scientific software ecosystem and its impact: current and future measures[J]. Research evaluation, 2015, 24(4): 454-470.
[22] LI K, CHEN P Y, YAN E. Challenges of measuring software impact through citations: an examination of the lme4 R package[J]. Journal of informetrics, 2019, 13(1): 449-461.
[23] PAN X, YAN E, WANG Q, et al. Assessing the impact of software on science: a bootstrapped learning of software entities in full-text papers[J]. Journal of informetrics, 2015, 9(4): 860-871.
[24] PAN X, YAN E, CUI M, et al. How important is software to library and information science research? A content analysis of full-text publications[J]. Journal of informetrics, 2019, 13(1): 397-406.
[25] ZHAO R, WEI M. Impact evaluation of open source software: an Altmetrics perspective[J]. Scientometrics, 2017, 110(2): 1017-1033.
[26] MA B, ZHANG C, WANG Y, et al. Enhancing identification of structure function of academic articles using contextual information[J]. Scientometrics, 2022, 127(2): 885-925.
[27] WANG Y, XIANG Y, ZHANG C. Exploring motivations for algorithm mention in the domain of natural language processing: a deep learning approach[J]. Journal of informetrics, 2024, 18(4): 101550.
[28] MITKOV R. The Oxford handbook of computational linguistics (Oxford handbooks) [M]. New York: Oxford University Press, 2005.
[29] MCHUGH M L. Interrater reliability: the kappa statistic[J]. Biochemia medica, 2012, 22(3): 276-282.
[30] VOOS H, DAGAEV K S. Are all citations equal? Or, did we op. cit. your idem?[J]. Journal of academic librarianship, 1976, 1(6): 19-21.
[31] HERLACH G. Can retrieval of information from citation indexes be simplified? multiple mention of a reference as a characteristic of the link between cited and citing article[J]. Journal of the American Society for Information Science, 1978, 29(6): 308-310.
[32] 夏红玉, 胡潜, 王忠义. 基于引文重要性的知识流动主路径分析[J]. 情报学报, 2022, 41(5): 451-462. (XIA H Y, HU Q, WANG Z Y. Tracing the knowledge flow main path based on important citations[J]. Journal of the China Society for Scientific and Technical Information, 2022, 41(5): 451-462.)
[33] MARICIC S, SPAVENTI J, PAVICIC L, et al. Citation context versus the frequency counts of citation histories[J]. Journal of the American Society for Information Science & technology, 1998, 49(6): 530-540.
[34] 李铮, 邓三鸿, 孔嘉, 等. 学者学术影响力识别研究——基于引文全数据的视角[J]. 图书情报工作, 2020, 64(12): 87-94. (LI Z, DENG S H, KONG J, et al. Research on scholar’s academic influence: from the perspective of full citation data [J]. Library and information service, 2020, 64(12): 87-94.)
[35] 周清清. 基于多源异构数据的图书影响力评价研究[D]. 南京: 南京理工大学, 2018. (ZHOU Q Q. Book impact assessment based on multi-source heterogeneous data[D]. Nanjing: Nanjing University of Science And Technology, 2018.)
[36] ZHANG C, ZHOU Q. Assessing books’ depth and breadth via multi-level mining on tables of contents[J]. Journal of informetrics, 2020, 14(2): 101032.
[37] 张光耀, 邵文君, 谢维熙, 等. 同行评议有效吗——基于有效市场假说的探究[J]. 图书情报知识, 2023, 40(2): 105-116. (ZHANG G Y, SHAO W J, XIE W X, et al. Is peer review effective? An exploration based on efficient market hypothesis [J]. Documentation, information & knowledge, 2023, 40(2): 105-116.)
[38] 张旭, 吕冬晴, 阮选敏, 等. 融合文献计量与同行评议共识的期刊评价方法研究[J]. 情报学报, 2022, 41(5): 486-496. (ZHANG X, LV D Q, RUAN X M, et al. Journal evaluation method based on the integration of peer-review consensus and bibliometric indicators [J]. Journal of the China Society for Scientific and Technical Information, 2022, 41(5): 486-496.)
[39] HOCHREITER S, SCHMIDHUBER J. Long Short-Term Memory[J]. Neural computation, 1997, 9(8): 1735-1780.
[40] LIN J. Divergence measures based on the Shannon entropy[J]. IEEE transactions on information theory, 1991, 37(1): 145-151.
Options
Outlines

/

〈 〉

    Copyrights © LIS Press Co., Ltd. Reproduction is prohibited without permission.