[目的/意义] 科研评价中,短时间引文窗口下的学科标准化指标往往是不可靠的,因为这时论文发表的时间较短,还没有充足的时间获取被引次数。然而,各种标准化方法本身并不能解决这一问题。研究旨在解决这一科研评价中的难题。[方法/过程] 研究引入一个权重因素以表示每篇论文标准分的可靠程度,权重由论文在给定的短时间窗口下的被引次数与长时间窗口下被引次数的相关系数计算获得,论文发表时间越短(长),可靠性越低(高),权重也越低(高)。为验证加权效果,将权重与常用的学科标准化指标CNCI进行加权处理,计算世界500强大学每所大学所有论文加权后的总影响力TWCNCI值与未加权时的总影响力TCNCI值。[结果/结论] 研究发现,500强大学的TWCNCI值与TCNCI值,TWCNCI的排名与TCNCI的排名都具有极强的相关性,但是仍有部分大学在加权后排名发生明显波动。据此,研究认为标准化指标在短时间窗口下不可靠的弊端以及对此修正的权重因素在科研评价中不应忽视。
[Purpose/significance] In the research evaluation, the field normalization indicator may not be sufficiently reliable when a short citation time window is used, because the publication time of the paper is shorter at this time, recent publications usually have insufficient time to accumulate the number of citations. However, all kinds of normalization methods themselves cannot solve this problem. [Method/process] This paper introduced a weighting factor representing the degree of reliability of the normalization citation count of one paper, which was calculated as the correlation coefficient between citation count of papers in the given short time window and those in the long time window. To verify the effect of the weighting, this paper introduced the weighting factor to weight the commonly used normalization indicator CNCI at the paper level and then computed the weighted total influence TWCNCI value and the unweighted total influence TCNCI value (Total CNCI) of all papers of each of the world’s top 500 universities. [Result/conclusion] The results show that although there was a strong correlation between the TWCNCI value and the TCNCI value and the rankings under TWCNCI and TCNCI of the world’s top 500 universities, some universities’ rankings have still changed significantly after weighting. This research demonstrates that the shortcomings of normalization indicators that are unreliable in a short time window and the weighting factors for this correction should not be ignored in the scientific research evaluation practices.
[1] RADICCHI F, FORTUNATO S, CASTELLANO C. Universality of citation distributions:toward an objective measure of scientific impact[J]. Proceedings of the National Academy of Sciences of the United States of America, 2008, 105(45):17268-17272.
[2] LUNDBERG J. Lifting the crown-citation z-score[J]. Journal of informetrics, 2007, 1(2):145-154.
[3] WALTMAN L, VAN ECK N J, VAN LEEUWEN T N, et al. Towards a new crown indicator:some theoretical considerations[J]. Journal of informetrics, 2011, 5(1):37-47.
[4] 伍军红, 肖宏, 任美亚, 等. PCSI:一种单篇论文被引频次标准化方法[J]. 图书情报工作, 2020, 64(23):22-30.
[5] BORNMANN L, WILLIAMS R. An evaluation of percentile measures of citation impact, and a proposal for making them better[J]. Scientometrics, 2020, 124(2):1457-1478.
[6] 刘雪立, 申蓝, 王燕, 等. 学术期刊的跨学科评价指标JIPR8及其应用[J]. 中国科技期刊研究, 2019, 30(6):663-670.
[7] RADICCHI F, CASTELLANO C. A reverse engineering approach to the suppression of citation biases reveals universal properties of citation distributions[J]. Plos one, 2012, 7(3):e33833.
[8] LEYDESDORFF L, OPTHOF T. Scopus's source normalized impact per paper (SNIP) versus a journal impact factor based on fractional counting of citations[J]. Journal of the American Society for Information Science and Technology, 2010, 61(11):2365-2369.
[9] WALTMAN L, VAN ECK N J. A systematic empirical comparison of different approaches for normalizing citation impact indicators[J]. Journal of informetrics, 2013, 7(4):833-849.
[10] ZITT M, SMALL H. Modifying the journal impact factor by fractional citation weighting:the audience factor[J]. Journal of the American Society for Information Science and Technology, 2008, 59(11):1856-1860.
[11] BORNMANN L. How can citation impact in bibliometrics be normalized? a new approach combining citing-side normalization and citation percentiles[J]. Quantitative science studies, 2020, 1(4):1553-1569.
[12] 李长玲, 刘运梅, 牌艳欣. 基于学科差别的论文影响力评价指标特征分析及体系构建[J]. 情报资料工作, 2019, 40(6):23-29.
[13] 俞立平, 张全, 刘爱军. 不同学科多属性评价横向比较研究--以数学、物理学、生物学期刊为例[J]. 图书情报工作, 2014, 58(20):100-105.
[14] HU Z G, TIAN W C, XU S M, et al. Four pitfalls in normalizing citation indicators:an investigation of ESI's selection of highly cited papers[J]. Journal of informetrics, 2018, 12(4):1133-1145.
[15] WALTMAN L, VAN ECK N J, VAN LEEUWEN T N, et al. Towards a new crown indicator:an empirical analysis[J].Scientometrics, 2011, 87(3):467-481.
[16] 陈福佑, 杨立英, 丁洁兰. 不同学科期刊学术影响力比较的方法与实证研究[J]. 图书情报工作, 2013, 57(23):85-89, 94.
[17] 陈仕吉, 史丽文, 李冬梅, 等. 论文被引频次标准化方法述评[J]. 现代图书情报技术, 2012, 28(4):54-60.
[18] 刘雪立, 申小曼, 郭佳, 等. 论文被引频次学科领域百分位在创建期刊百分位数指标中的应用[J]. 中国科技期刊研究, 2021, 32(1):118-124.
[19] 任元秋, 王 兴, 郑钦钦. 不同学科分类方案下不同学科标准化方法效果的比较研究[J]. 图书情报工作, 2021, 65(3):84-92.
[20] 宋丽萍, 王建芳. 基于学科规范引文影响力与同行评议相关性的科学评价实证研究[J]. 图书情报工作, 2018, 62(18):122-128.
[21] 张志辉, 程莹, 刘念才. 线性学科标准化方法的效果优化及其对科研评价结果的影响--以39所"985工程"大学论文质量排名为例[J]. 情报学报, 2015, 34(3):300-312.
[22] WANG J. Citation time window choice for research impact evaluation[J]. Scientometrics, 2013, 94(3):851-872.
[23] WANG X, ZHANG Z H. Improving the reliability of short-term citation impact indicators by taking into account the correlation between short- and long-term citation impact[J]. Journal of informetrics, 2020, 14(2):101019.
[24] WANG X. The relationship between SCI editorial board representation and university research output in the field of computer science:a quantile regression approach[J]. Malaysian journal of library and information science, 2018, 23(1):67-84.
[25] 王兴. 国际学术期刊编委数量与科研产出评价指标的相关性研究--以经济学学科世界984所大学为例[J]. 重庆大学学报(社会科学版), 2017, 23(1):61-70.
[26] ABRAMO G, D'ANGELO C A, FELICI G. Predicting publication long-term impact through a combination of early citations and journal impact factor[J]. Journal of informetrics, 2019, 13(1):32-49.
[27] BORNMANN L, LEYDESDORFF L, WANG J. How to improve the prediction based on citation impact percentiles for years shortly after the publication date? [J]. Journal of informetrics, 2014, 8(1):175-180.
[28] WANG D, SONG C, BARABASI A L. Quantifying long-term scientific impact[J]. Science, 2013, 342(6154):127-132.
[29] TAHAMTAN I, AFSHAR A S, AHAMDZADEH K. Factors affecting number of citations:a comprehensive review of the literature[J]. Scientometrics, 2016, 107(3):1195-1225.
[30] BAI X, ZHANG F, LEE I. Predicting the citations of scholarly paper[J]. Journal of informetrics, 2019, 13(1):407-418.
[31] KOSTEAS V D. Predicting long-run citation counts for articles in top economics journals[J]. Scientometrics, 2018, 115(3):1395-1412.
[32] THELWALL M, NEVILL T. Could scientists use Altmetric.com scores to predict longer term citation counts? [J]. Journal of informetrics, 2018, 12(1):237-248.
[33] WANG M, WANG Z, CHEN G. Which can better predict the future success of articles? Bibliometric indices or alternative metrics[J]. Scientometrics, 2019, 119(3):1575-1595.
[34] MOED H F, DE BRUIN R E, VAN LEEUWEN T N. New bibliometric tools for the assessment of national research performance:database description, overview of indicators and first applications[J]. Scientometrics, 1995, 33(3):381-422.
[35] CWTS. CWTS Leiden Ranking 2019 [EB/OL]. [2021-05-04]. https://www.leidenranking.com/ranking/2019/list.
[36] 胡泽文, 武夷山, 袁军鹏. 典型的高校排名做法与借鉴意义[J]. 高教发展与评估, 2016, 32(2):27-37, 99.
