[Purpose/Significance] Major advances in science depend largely on breakthrough discoveries. As the main carrier of breakthrough discovery, the identification and discovery of breakthrough papers play an important role in science and technology policy formulation and basic research cultivation. [Method/Process] At present, the quantitative identification methods have problems in using a single feature dimension and lacking close connection with the scientific breakthrough. This paper combines the features of breakthrough research in the biomedical field to propose a breakthrough paper identification method that can more comprehensively reflect the laws and features of breakthrough innovation. Firstly, the indicators are constructed from the perspectives of knowledge innovation, diffusion, dissemination path, integration, uncertainty and controversy to measure the novelty, influence, scientific destructiveness, interdisciplinary feature and controversial feature of breakthrough papers. Then the random forest model is applied and the feature indicators are used as input variables to construct a breakthrough identification model in the biomedical field. Finally, the model is evaluated and the parameters are tuned. [Result/Conclusion] The F1 value of the breakthrough paper identification model in the biomedical field is 0.834 1, and the indicators of influence, controversy and scientific destructive features have a great influence on the identification results of the model. In the new sample of selected papers by Lasker Prize winners and milestone literature in the field of oncology, the model maintains relatively stable identification performance and good generalization performance.
[1] NIH. NIH director’s transformative research award[EB/OL]. [2024-04-07]. https://commonfund.nih.gov/tra/award-overview.
[2] XI J. Secure a decisive victory in building a moderately prosperous society in all respects and strive for the great success of socialism with Chinese characteristics for a new era[J]. Beijing review, 2017, 60(46): 1-9.
[3] HYRYNEN M. Breakthrough research: funding for high-risk research at the academy of Finland[R]. Helsinki: Academy of Finland, 2007.
[4] HOLLINGSWORTH J R. Scientific discoveries: an institutionalist and path-dependent perspective[M]. Amsterdam: Biomedicine in the twentieth century: practices, policies, and politics, 2008: 317-353.
[5] Science. Science newsletters[EB/OL]. [2024-04-07]. http://www.sciencemagchina.cn/highlights141219.aspx.
[6] CHEN C, SONG M, HEO G E. A scalable and adaptive method for finding semantically equivalent cue words of uncertainty[J]. Journal of informetrics, 2018, 12(1): 158-180.
[7] MURRAY D, LAMERS W, BOYACK K, et al. Measuring disagreement in Science[C]//Proceedings of the 17th international conference of the International Society for Scientometrics and Informetrics. Rome: ISSI, 2019: 2370-2375
[8] 杜建.医学知识不确定性测度的进展与展望[J]. 数据分析与知识发现, 2020, 4(10): 14-27. (DU J. Measuring uncertainty of medical knowledge: a literature review[J]. Data analysis and knowledge discovery, 2020, 4(10): 14-27.)
[9] TREVORS J T, POLLACK G H, SAIER M H, et al. Transformative research: definitions, approaches and consequences[J]. Theory in biosciences, 2012, 131(2): 117-123.
[10] KOSHLAND D E. The Cha-Cha-Cha theory of scientific discovery[J]. Science, 2007, 317(5839): 761–762.
[11] WINNINK J J, TIJSSEN R J W, VAN RAAN A F J. Searching for new breakthroughs in science: how effective are computerised detection algorithms?[J]. Technological forecasting and social change, 2018, 146: 673-686.
[12] PONOMAREV I V, WILLIAMS D E, HACKETT C J, et al. Predicting highly cited papers: a method for early detection of candidate breakthroughs[J]. Technological forecasting and social change, 2014, 81: 49-55.
[13] 迟培娟, 丁洁兰, 冷伏海.突破性论文的三元计量特征及识别研究——以生物医学领域为例[J]. 情报学报, 2022, 41(7): 663-675. (CHI P J, DING J L, LENG F H. Characteristics of the technology impact of breakthrough papers in biology and medicine[J]. Journal of the China Society for Scientific and Technical Information, 2022, 41(7): 663-675.)
[14] HUANG Y H, HSU C N, LERMAN K. Identifying transformative scientific research[C]//2013 IEEE 13th international conference on data mining. Dallas: ICDM, 2013: 291-300.
[15] 杜建, 孙轶楠, 张阳, 等.变革性研究的科学计量学特征与早期识别方法[J]. 中国科学基金, 2019, 33(1): 88-98. (DU J, SUN Y N, ZHANG Y, et al. Characterizing and detecting the early scientometric sign of the potential transformative research[J]. Science Foundation in China, 2019, 33(1): 88-98.)
[16] 王雪.基于引文全文本的医学领域突破性文献识别研究[D]. 北京: 北京协和医学院, 2022. (WANG X. Research on identification of medical breakthrough articles based on citing sentences[D]. Beijing: Peking Union Medical College, 2022.)
[17] CHAI S, MENON A. Breakthrough recognition: bias against novelty and competition for attention[J]. Research policy, 2019, 48(3): 733-747.
[18] MISHRA S, TORVIK V I. Quantifying conceptual novelty in the biomedical literature[J]. D-lib magazine: the magazine of the digital library forum, 2016, 22: 9-10.
[19] 张金柱, 张晓林.基于被引科学知识主题突变的突破性创新识别[J]. 现代图书情报技术, 2016(Z1): 42-50. (ZHANG J Z, ZHANG X L. Radical innovation identification based on topic mutation of scientific knowledge cited in patents[J]. Data analysis and knowledge discovery, 2016(Z1): 42-50.)
[20] 梁国强. 变革性研究的引文特征分析与早期识别模型构建[D]. 大连: 大连理工大学, 2021. (LIANG G Q. Citation patterns analysis and the early identification model construction of transformative research[D]. Dalian: Dalian University of Technology, 2021.)
[21] PONOMAREV I V, LAWTON B K, WILLIAMS D E, et al. Breakthrough paper indicator 2.0: can geographical diversity and interdisciplinarity improve the accuracy of outstanding papers prediction?[J]. Scientometrics, 2014, 100(3): 755-765.
[22] SCHNEIDER J W, COSTAS R. Identifying potential “breakthrough” publications using refined citation analyses: three related explorative approaches[J]. Journal of the Association for Information Science and Technology, 2017, 68(3): 709-723.
[23] PONOMAREV I V, WILLIAMS D E, LAWTON B K, et al. Breakthrough paper indicator: early detection and measurement of ground-breaking research[C]//11th international conference on CRIS. Prague: CRIS, 2012: 295-304.
[24] MARIANI M S, MEDO M, ZHANG Y C. Identification of milestone papers through time-balanced network centrality[J]. Journal of informetrics, 2016, 10(4): 1207-1223.
[25] CHEN C. Predictive effects of structural variation on citation counts[J]. Journal of the American Society for Information Science and Technology, 2012, 63(3): 431-449.
[26] WU L, WANG D, EVANS J A. Large teams have developed science and technology; small teams have disrupted it[J]. Social science electronic publishing, 2017, 566(7744): 378-382.
[27] BORNMANN L, TEKLES A. Convergent validity of several indicators measuring disruptiveness with milestone assignments to physics papers by experts[J]. Journal of informetrics, 2021, 15(3): 101-159.
[28] BORNMANN L, DEVARAKONDA S, TEKLES A, et al. Are disruption index indicators convergently valid? the comparison of several indicator variants with assessments by peers[J]. Quantitative science studies, 2020, 1(4): 1-31.
[29] WOLCOTT H N, FOUCH M J, HSU E R, et al. Modeling time-dependent and -independent indicators to facilitate identification of breakthrough research papers[J]. Scientometrics, 2016, 107(2): 807-817.
[30] FU L D, ALIFERIS C F. Using content-based and bibliometric features for machine learning models to predict citation counts in the biomedical literature[J]. Scientometrics, 2010, 85(1): 257-270.
[31] LIAW A, WEINER M. Classification and regression by random forest[J]. R news, 2002, 2:18-22.
[32] 林紫洛. 基于双重特征的突破性论文识别研究[D]. 北京: 北京协和医学院, 2023. (LIN Z L. Research on identification of breakthrough papers based on dual features[D]. Beijing: Peking Union Medical College, 2023.)
[33] 王康, 陈悦.基于异质性专利的颠覆性技术早期识别研究[J]. 科学学研究, 2023, 41(8): 1364-1375. (WANG K, CHEN Y. Research on early identification of disruptive technologies based on heterogeneous patents[J]. Studies in Science of Science, 2023, 41(8): 1364-1375.)
[34] WANG J. Citation time window choice for research impact evaluation[J]. Scientometrics, 2013, 94(3): 851-872.
[35] HARGADON A. How breakthroughs happen: the surprising truth about how companies innovate[M]. Boston: Harvard Business Review Press, 2003.
[36] STIRLING A. A general framework for analysing diversity in science, technology and society[J]. Journal of the royal society interface, 2007, 4(15): 707-719.
[37] ZHANG L, LEYDESDORFF L. The scientometric measurement of interdisciplinarity and diversity in the research portfolios of Chinese universities[J]. Journal of data and information science, 2021, 6(4): 13-35.
[38] 杜建.面向复杂决策和知识发现的医学知识不确定性计算方法[J]. 医学信息学杂志, 2022, 43(7): 32-38. (DU J. Approaches on measuring the uncertainty of medical knowledge for complex decision making and knowledge discovery[J]. Journal of medical informatics, 2022, 43(7): 32-38.)
[39] CHARLTON B G. Measuring revolutionary biomedical science 1992-2006 using Nobel prizes, Lasker (clinical medicine) awards and Gairdner awards (NLG metric)[J]. Medical hypotheses, 2007, 69(1): 1-5.
[40] BROOM F E. Breakthroughs of the year, 1996[J]. Science, 1996, 274(5295): 1987.
[41] WANG X, YANG X, DU J, et al. A deep learning approach for identifying biomedical breakthrough discoveries using context analysis[J]. Scientometrics, 2021, 126(3): 5531-5549.
[42] LI J, YIN Y, FORTUNATO S, et al. A dataset of publication records for Nobel laureates[J]. Scientific data, 2019, 6(1): 33-42.
[43] AHLGREN P, COLLIANDER C, SJGRDE P. Exploring the relation between referencing practices and citation impact: a large‐scale study based on Web of Science data[J]. Journal of the Association for Information Science and Technology, 2018, 69(5): 728-743.
[44] Nature. Milestones in cancer[EB/OL]. [2024-04-07]. https://www.nature.com/immersive/d42859-020-00083-8/index.html.
