[Purpose/Significance] This study comparatively analyzes the effectiveness and limitations of two mainstream methodologies (reference-based and topic content-based approaches) to provide insights for constructing a multi-dimensional collaborative evaluation framework for knowledge novelty and references for developing a unified measurement approach for assessing the value of knowledge innovation. [Method/Process] This study employed atypical combination approach (reference dimension) by B. Uzzi et al. and topic-year prediction method (topic content dimension) by P. Savov. Through kernel density distribution analysis, correlation analysis (Spearman’s test), and multivariate regression models, it systematically validated the representational capacity of these two methods for knowledge novelty. Case studies further elucidated their intrinsic mechanistic differences. [Result/Conclusion] Key findings are as follows. ①Consistency in kernel density distributions and case studies of literature abstracts confirm the partial validity of both methods. Annual variations in their correlations with citation counts and textual interpretations reveal their tendencies to capture distinct dimensions of knowledge novelty. ②While the atypical combination method is influenced by the citation frequency of reference sources, its identification of highly innovative sources demonstrates validity for combination analysis. The topic-year prediction method, though significantly affected by temporal factors, exhibits validity in identifying novelty within the same year. ③Annual variations in mean innovation values and cumulative density distributions across stages further corroborate the pronounced temporal bias of the topic-year prediction method. In factor analysis, citation count exerts a strong positive influence (0.495) on the topic-year prediction method, highlighting its temporal dependency, while the number of reference sources positively impacts the atypical combination method (0.193).
[1] SCHNEIDER J, COSTAS R. Identifying potential “breakthrough” publications using refined citation analyses: three related explorative approaches[J]. Journal of the Association for Information Science and Technology, 2017, 68(3): 709-723.
[2] MIN C, DING Y, LI J, et al. Innovation or imitation: the diffusion of citations[J]. Journal of the Association for Information Science and Technology, 2018, 69(10): 1271-1282.
[3] MUKHERJEE S, ROMERO D, JONES B, et al. The nearly universal link between the age of past knowledge and tomorrow’s breakthroughs in science and technology: the hotspot[J]. Science advances, 2017, 3(4): e1601315.
[4] UZZI B, MUKHERJEE S, STRINGER M, et al. Atypical combinations and scientific impact[J]. Science, 2013, 342(6157): 468-472.
[5] SHIBAYAMA S, WANG J. Measuring originality in science[J]. Scientometrics, 2020, 122(1): 409-427.
[6] LI X, WEN Y, JIANG J, et al. Identifying potential breakthrough research: a machine learning method using scientific papers and Twitter data[J]. Technological forecasting and social change, 2022, 184: 122042.
[7] YAN Y, TIAN S, ZHANG J. The impact of a paper’s new combinations and new components on its citation[J]. Scientometrics, 2020, 122(2): 895-913.
[8] HOFSTRA B, KULKARNI V, GALVEZ S, et al. The diversity-innovation paradox in science[J]. Proceedings of the National Academy of Sciences, 2020, 117(17): 9284-9291.
[9] MIN C, BU Y, SUN J. Predicting scientific breakthroughs based on knowledge structure variations[J]. Technological forecasting and social change, 2021, 164: 120502.
[10] MIN C, BU Y, WU D, et al. Identifying citation patterns of scientific breakthroughs: a perspective of dynamic citation process[J]. Information processing & management, 2021, 58(1): 102428.
[11] XU H, LUO R, WINNINK J, et al. A methodology for identifying breakthrough topics using structural entropy[J]. Information processing & management, 2022, 59(2): 102862.
[12] WANG S, MA Y, MAO J, et al. Quantifying scientific breakthroughs by a novel disruption indicator based on knowledge entities[J]. Journal of the Association for Information Science and Technology, 2023, 74(2): 150-167.
[13] 温浩, 何茜茹. 学术文摘创新点挖掘的认知分析方法[J]. 情报学报, 2021, 40(5): 489-499. (WEN H, HE Q R. Cognitive analysis method for mining innovation points in academic abstracts[J]. Journal of the China Society for Scientific and Technical Information, 2021, 40(5): 489-499.)
[14] 曹树金,闫欣阳,张倩等. 中外情报学论文创新性特征研究[J]. 图书情报工作, 2020, 64(1): 80-92. (CAO S J, YAN X Y, ZHANG Q, et al. Research on characteristics of innovation in Chinese and international academic literature of information science[J]. Library and information service, 2020, 64(1): 80-92.)
[15] SUN B, KOLESNIKOV S, GOLDSTEIN A, et al. A dynamic approach for identifying technological breakthroughs with an application in solar photovoltaics[J]. Technological forecasting and social change, 2021, 165: 120534.
[16] LUO Z, LU W, HE J, et al. Combination of research questions and methods: a new measurement of scientific novelty[J]. Journal of informetrics, 2022, 16(2): 101282.
[17] SAVOV P, JATOWT A & NIELEK R. Identifying breakthrough scientific papers[J]. Information processing & management, 2020, 57(2): 102168.
[18] HICKS D, WOUTERS P, WALTMAN L, et al. Bibliometrics: the Leiden Manifesto for research metrics[J]. Nature, 2015, 520(7548): 429-431.
[19] WU L, KITTUR A, YOUN H, et al. Metrics and mechanisms: measuring the unmeasurable in the science of science[J]. Journal of informetrics, 2022, 16(2): 101290.
[20] WAGNER C, WHETSELL T, MUKHERJEE S. International research collaboration: novelty, conventionality, and atypicality in knowledge recombination[J]. Research policy, 2019, 48(5): 1260-1270.
[21] BOUDREAU K, GUINAN E, LAKHANI K, et al. Looking across and looking beyond the knowledge frontier: intellectual distance, novelty, and resource allocation in science[J]. Management science, 2016, 62(10): 2765-2783.
[22] FONTANA M, IORI M, MONTOBBIO F, et al. New and atypical combinations: an assessment of novelty and interdisciplinarity[J]. Research policy, 2020, 49(7): 104063.
[23] BORNMANN L, TEKLES A. Convergent validity of several indicators measuring disruptiveness with milestone assignments to physics papers by experts[J]. Journal of informetrics, 2021, 15(3): 101159.
[24] ABBASIHAROFTEH M, KOGLER D, LENGYEL B. Atypical combinations of technologies in regional co-inventor networks[J]. Research policy, 2023, 52(10): 104886.
[25] MUKHERJEE S, UZZI B, JONES B, et al. A new method for identifying recombinations of existing knowledge associated with high-impact innovation[J]. Journal of product innovation management, 2016, 33(2): 224-336.
