Early Identification Methods for Potential Breakthrough Papers Based on Binary Quantitative Features

Dou Yongxiang; Jiang Mengfan; Zhai Jiarui; Wang Jiamin

doi:10.13266/j.issn.0252-3116.2025.15.005

Library and Information Service >

2025 , Vol. 69 >Issue 15: 53 - 63

DOI: https://doi.org/10.13266/j.issn.0252-3116.2025.15.005

RESEARCH PAPERS

Early Identification Methods for Potential Breakthrough Papers Based on Binary Quantitative Features

Dou Yongxiang ,
Jiang Mengfan ,
Zhai Jiarui ,
Wang Jiamin

Expand

School of Economics and Management, Xidian University, Xi'an 710126

Dou Yongxiang, professor, PhD, doctoral supervisor, E-mail: yxdou@xidian.edu.cn; Jiang Mengfan, master’s degree; Zhai Jiarui, master candidate; Wang Jiamin, lecturer, PhD, master’s supervisor.

Received date: 2024-05-15

Revised date: 2024-10-08

Online published: 2025-07-21

Supported by

This work is supported by the National Natural Science Foundation of China project titled “Research on the Identification and Evolution Path of Interdisciplinary Knowledge Structure Based on Text Semantic Understanding” (Grant No. 72304218) and by the Fundamental Research Funds for the Central Universities of Ministry of Education of China, titled “Research on New Generation Information Technology Strategy”(Grant No. KYFZ24069).

Fold

Abstract

[Purpose/Significance] Potential breakthrough papers are the most promising “seeds” in basic research, potentially indicating transformations in the scientific, technological, and innovative systems. Identifying breakthrough papers as early as possible can help national scientific research management departments fund potential disruptive basic researches and achieve forward-looking strategic deployment in scientific and technological innovation. [Method/Process] Firstly, based on the theory of scientific revolution and the connotation of breakthrough papers, it pointed out that breakthrough papers had two main characteristics: disruptiveness and influence. Then, with the goal of early identification, measurement indicators for disruptive characteristics were selected by considering both significant differences and index stability. By utilizing the XGBoost model, the prediction of papers’ future influence characteristics was made, and papers that simultaneously met the thresholds of high disruptiveness and high influence were regarded as potential breakthrough papers. Finally, the brain science field was taken as an example for verification. [Result/Conclusion] The method proposed in this paper can effectively carry out early identification of potential breakthrough papers, but whether this method has domain transferability or not remains to be further studied.

Key words： scientific breakthrough; breakthrough paper; disruptive index; machine learning method; early identification

Cite this article

Dou Yongxiang , Jiang Mengfan , Zhai Jiarui , Wang Jiamin . Early Identification Methods for Potential Breakthrough Papers Based on Binary Quantitative Features[J]. Library and Information Service, 2025 , 69(15) : 53 -63 . DOI: 10.13266/j.issn.0252-3116.2025.15.005

References

[1] National Science Board 2020 Vision for the National Science Foundation, NSB-05-142[R]. Washington, D.C.: National Science Foundation, 2005.
[2] 国务院. 国务院关于印发“十三五”国家科技创新规划的通知[EB/OL]. [2025-06-09]. https://www.gov.cn/zhengce/content/ 2016-08/08/content_5098072.htm. (State council. Circular of the state council on printing and issuing the '13th five-year' national science and technology innovation plan[EB/OL]. [2025-06-09]. https://www.gov.cn/zhengce/content/2016-08/08/content_5098072.htm.)
[3] European Commission. 地平线欧洲研究和资助计划[EB/OL]. [2025-06-09]. https://research-and-innovation.ec.europa.eu/funding/funding-opportunities/funding-programmes-and-open-calls/horizon-europe_en. (European Commission. Horizon Europe[EB/OL]. [2025-06-09]. https://research-and-innovation.ec.europa.eu/funding/funding-opportunities/funding-programmes-and-open-calls/horizon-europe_en.)
[4] SCHNEIDER J W, COSTAS R. Identifying potential “breakthrough” publications using refined citation analyses: three related explorative approaches[J]. Journal of the Association for Information Science and Technology, 2017, 68(3): 709-723.
[5] WOLCOTT H N, FOUCH M J, HSU E R, et al. Modeling time-dependent and -independent indicators to facilitate identification of breakthrough research papers[J]. Scientometrics, 2016, 107(2): 807-817.
[6] 杜建, 孙轶楠, 张阳, 等. 变革性研究的科学计量学特征与早期识别方法[J]. 中国科学基金, 2019, 33(1): 88-98. (DU J, SUN Y N, ZHANG Y, et al. Characterizing and detecting the early scientometric sign of the potential transformative research[J]. Science foundation in China, 2019, 33(1): 88-98.)
[7] 梁国强. 变革性研究的引文特征分析与早期识别模型构建[D]. 大连: 大连理工大学, 2021. (LIANG G Q. Citation patterns analysis and the early identification model construction of transformative research[D]. Dailan: Dalian University of Technology, 2021.)
[8] 郭丽娜, 李星琛, 左文革, 等. 颠覆性研究文献计量识别方法述评[J]. 数字图书馆论坛, 2020(3): 17-24. (GUO L N, LI X C, ZUO W G. A review of disruptive work and its bibliometric identification methods[J]. Digital library forum, 2020(3): 17-24.)
[9] MIN C, BU Y, SUN J. Predicting scientific breakthroughs based on knowledge structure variations[J]. Technological forecasting and social change, 2021, 164: 120502.
[10] WANG S, MA Y, MAO J, et al. Quantifying scientific breakthroughs by a novel disruption indicator based on knowledge entities [J]. Journal of the Association for Information Science and Technology, 2023, 74(2): 150-167.
[11] 王海燕, 潘云涛, 马峥, 等. 基于科学研究问题成熟度的未来高影响力科技论文预测研究[J]. 情报学报, 2016, 35(1): 36-47. (WANG H Y, PAN Y T, MA Z, et al. Prediction study on articles with future high-impact based on study readiness level[J]. Journal of the China Society for Scientific and Technical Information, 2016, 35(1): 36-47.)
[12] KUHN T S. The structure of scientific revolution: Second Edition[M]. Chicago: University of Chicago Press, 1970.
[13] 迟培娟, 丁洁兰, 冷伏海. 突破性论文的三元计量特征及识别研究[J]. 情报学报, 2022, 41(7): 663-675. (CHI P J, DING J L, LENG F H. Characteristics of the technology impact of breakthrough papers in biology and medicine[J]. Journal of the China Society for Scientific and Technical Information, 2022, 41(7): 663-675.)
[14] BORNMANN L, TEKLES A. Disruptive papers published in Scientometrics[J]. Scientometrics, 2019, 120(1): 331-336.
[15] WINNINK J J, TIJSSEN R J W, VAN RAAN A F J. Searching for new breakthroughs in science: how effective are computerised detection algorithms[J]. Technological forecasting and social change, 2019, 146: 673-686.
[16] PONOMAREV I V, WILLIAMS D E, HACKETT C J, et al. Predicting highly cited papers: a method for early detection of candidate breakthroughs[J]. Technological forecasting and social change, 2014, 81: 49-55.
[17] WINNINK J J, TIJSSEN R J W. Early stage identification of breakthroughs at the interface of science and technology: lessons drawn from a landmark publication [J]. Scientometrics, 2015, 102(1): 113-134.
[18] XU H, WINNINK J, YUE Z H, ZHANG H L, PANG H S, Multidimensional scientometric indicators for the detection of emerging research topics[J]. Technological forecasting and social change, 2021, 163: 120490.
[19] MIN C, BU Y, WU D, et al. Identifying citation patterns of scientific breakthroughs: a perspective of dynamic citation process[J]. Information processing & management, 2021, 58(1): 10242.
[20] MIN C, BU Y, SUN J, et al. Is scientific novelty reflected in citation patterns[J]. Proceedings of the Association for Information Science and Technology, 2018, 55(1): 875-876.
[21] FUNK R J, OWEN-SMITH J. A dynamic network measure of technological change[J]. Management science, 2017, 63(3): 791-817.
[22] WU L, WANG D, EVANS J A. Large teams develop and small teams disrupt science and technology [J]. Nature, 2019, 566(7744): 378-382.
[23] 刘小慧, 沈哲思, 廖宇, 等. 科研论文颠覆性指数的改进及其影响因素研究[J]. 图书情报工作, 2020, 64(24): 84-91. (LIU X H, SHEN Z S, LIAO Y, et al. The research about the improved disruption index and its influencing factors[J]. Library and information service, 2020, 64(24): 84-91.)
[24] CASADEVALL A, FANG F C. Revolutionary science[J]. mBio, 2016, 7(2): e00158.
[25] 杜建, 武夷山. 睡美人与王子文献的识别方法研究[J]. 图书情报工作, 2015, 59(19): 84-92. (DU J, WU Y S. Defining and identifying the sleeping beauty and her princes in science[J]. Library and information service, 2015, 59(19): 84-92.)
[26] WANG D, SONG C, BARABÁSI A L. Quantifying long-term scientific impact[J]. Science, 2013, 342(6154): 127-132.
[27] SAVOV P, JATOWT A, NIELEK R. Identifying breakthrough scientific papers[J]. Information processing & management, 2020, 57(2): 102168.
[28] BORNMANN L, DEVARAKONDA S, TEKLES A, et al. Are disruption index indicators convergently valid? the comparison of several indicator variants with assessments by peers[J]. Quantitative science studies, 2020, 1(3): 1242-1259.
[29] BORNMANN L, TEKLES A. Disruption index depends on length of citation window[J]. El profesional de la información, 2019, 28(2): e280207.
[30] CHARLTON B G. Scientometric identification of elite ‘revolutionary science’ research institutions by analysis of trends in Nobel prizes 1947–2006[J]. Medical hypotheses, 2007, 68(5): 931-934.
[31] WEI C, LI J, SHI D. Quantifying revolutionary discoveries: evidence from Nobel prize-winning papers[J]. Information processing & management, 2023, 60(3): 103252.
[32] HASLAM N, BAN L, KAUFMANN L, et al. What makes an article influential? Predicting impact in social and personality psychology[J]. Scientometrics, 2008, 76(1): 169-185.
[33] KOSTOFF R. The difference between highly and poorly cited medical articles in the journal Lancet[J]. Scientometrics, 2007, 72(3): 513-520.
[34] YU T, YU G, LI P Y, et al. Citation impact prediction for scientific papers using stepwise regression analysis[J]. Scientometrics, 2014, 101: 1233-1252.
[35] BJARNASON T, SIGFUSDOTTIR I D. Nordic impact: article productivity and citation patterns in sixteen Nordic sociology departments[J]. Acta sociologica, 2002, 45(4): 253-267.
[36] DIDEGAH F, THELWALL M. Determinants of research citation impact in nanoscience and nanotechnology[J]. Journal of the American Society for Information Science and Technology, 2013, 64(5): 1055-1064.
[37] WANG J. Citation time window choice for research impact evaluation[J]. Scientometrics, 2013, 94(3): 851-872.
[38] KENKEL W. Corpus colossal: a bibliometric analysis of neuroscience abstracts and impact factor[J]. Frontiers in integrative neuroscience, 2019, 13: 18.
[39] NEWMAN M E J. Prediction of highly cited papers[J]. Europhysics letters, 2014, 105(2): 28002.

Options

Outlines

模态框（Modal）标题

Abstract

Cite this article

References