[Purpose/significance] In the research evaluation, the field normalization indicator may not be sufficiently reliable when a short citation time window is used, because the publication time of the paper is shorter at this time, recent publications usually have insufficient time to accumulate the number of citations. However, all kinds of normalization methods themselves cannot solve this problem. [Method/process] This paper introduced a weighting factor representing the degree of reliability of the normalization citation count of one paper, which was calculated as the correlation coefficient between citation count of papers in the given short time window and those in the long time window. To verify the effect of the weighting, this paper introduced the weighting factor to weight the commonly used normalization indicator CNCI at the paper level and then computed the weighted total influence TWCNCI value and the unweighted total influence TCNCI value (Total CNCI) of all papers of each of the world’s top 500 universities. [Result/conclusion] The results show that although there was a strong correlation between the TWCNCI value and the TCNCI value and the rankings under TWCNCI and TCNCI of the world’s top 500 universities, some universities’ rankings have still changed significantly after weighting. This research demonstrates that the shortcomings of normalization indicators that are unreliable in a short time window and the weighting factors for this correction should not be ignored in the scientific research evaluation practices.
[1] RADICCHI F, FORTUNATO S, CASTELLANO C. Universality of citation distributions:toward an objective measure of scientific impact[J]. Proceedings of the National Academy of Sciences of the United States of America, 2008, 105(45):17268-17272.
[2] LUNDBERG J. Lifting the crown-citation z-score[J]. Journal of informetrics, 2007, 1(2):145-154.
[3] WALTMAN L, VAN ECK N J, VAN LEEUWEN T N, et al. Towards a new crown indicator:some theoretical considerations[J]. Journal of informetrics, 2011, 5(1):37-47.
[4] 伍军红, 肖宏, 任美亚, 等. PCSI:一种单篇论文被引频次标准化方法[J]. 图书情报工作, 2020, 64(23):22-30.
[5] BORNMANN L, WILLIAMS R. An evaluation of percentile measures of citation impact, and a proposal for making them better[J]. Scientometrics, 2020, 124(2):1457-1478.
[6] 刘雪立, 申蓝, 王燕, 等. 学术期刊的跨学科评价指标JIPR8及其应用[J]. 中国科技期刊研究, 2019, 30(6):663-670.
[7] RADICCHI F, CASTELLANO C. A reverse engineering approach to the suppression of citation biases reveals universal properties of citation distributions[J]. Plos one, 2012, 7(3):e33833.
[8] LEYDESDORFF L, OPTHOF T. Scopus's source normalized impact per paper (SNIP) versus a journal impact factor based on fractional counting of citations[J]. Journal of the American Society for Information Science and Technology, 2010, 61(11):2365-2369.
[9] WALTMAN L, VAN ECK N J. A systematic empirical comparison of different approaches for normalizing citation impact indicators[J]. Journal of informetrics, 2013, 7(4):833-849.
[10] ZITT M, SMALL H. Modifying the journal impact factor by fractional citation weighting:the audience factor[J]. Journal of the American Society for Information Science and Technology, 2008, 59(11):1856-1860.
[11] BORNMANN L. How can citation impact in bibliometrics be normalized? a new approach combining citing-side normalization and citation percentiles[J]. Quantitative science studies, 2020, 1(4):1553-1569.
[12] 李长玲, 刘运梅, 牌艳欣. 基于学科差别的论文影响力评价指标特征分析及体系构建[J]. 情报资料工作, 2019, 40(6):23-29.
[13] 俞立平, 张全, 刘爱军. 不同学科多属性评价横向比较研究--以数学、物理学、生物学期刊为例[J]. 图书情报工作, 2014, 58(20):100-105.
[14] HU Z G, TIAN W C, XU S M, et al. Four pitfalls in normalizing citation indicators:an investigation of ESI's selection of highly cited papers[J]. Journal of informetrics, 2018, 12(4):1133-1145.
[15] WALTMAN L, VAN ECK N J, VAN LEEUWEN T N, et al. Towards a new crown indicator:an empirical analysis[J].Scientometrics, 2011, 87(3):467-481.
[16] 陈福佑, 杨立英, 丁洁兰. 不同学科期刊学术影响力比较的方法与实证研究[J]. 图书情报工作, 2013, 57(23):85-89, 94.
[17] 陈仕吉, 史丽文, 李冬梅, 等. 论文被引频次标准化方法述评[J]. 现代图书情报技术, 2012, 28(4):54-60.
[18] 刘雪立, 申小曼, 郭佳, 等. 论文被引频次学科领域百分位在创建期刊百分位数指标中的应用[J]. 中国科技期刊研究, 2021, 32(1):118-124.
[19] 任元秋, 王 兴, 郑钦钦. 不同学科分类方案下不同学科标准化方法效果的比较研究[J]. 图书情报工作, 2021, 65(3):84-92.
[20] 宋丽萍, 王建芳. 基于学科规范引文影响力与同行评议相关性的科学评价实证研究[J]. 图书情报工作, 2018, 62(18):122-128.
[21] 张志辉, 程莹, 刘念才. 线性学科标准化方法的效果优化及其对科研评价结果的影响--以39所"985工程"大学论文质量排名为例[J]. 情报学报, 2015, 34(3):300-312.
[22] WANG J. Citation time window choice for research impact evaluation[J]. Scientometrics, 2013, 94(3):851-872.
[23] WANG X, ZHANG Z H. Improving the reliability of short-term citation impact indicators by taking into account the correlation between short- and long-term citation impact[J]. Journal of informetrics, 2020, 14(2):101019.
[24] WANG X. The relationship between SCI editorial board representation and university research output in the field of computer science:a quantile regression approach[J]. Malaysian journal of library and information science, 2018, 23(1):67-84.
[25] 王兴. 国际学术期刊编委数量与科研产出评价指标的相关性研究--以经济学学科世界984所大学为例[J]. 重庆大学学报(社会科学版), 2017, 23(1):61-70.
[26] ABRAMO G, D'ANGELO C A, FELICI G. Predicting publication long-term impact through a combination of early citations and journal impact factor[J]. Journal of informetrics, 2019, 13(1):32-49.
[27] BORNMANN L, LEYDESDORFF L, WANG J. How to improve the prediction based on citation impact percentiles for years shortly after the publication date? [J]. Journal of informetrics, 2014, 8(1):175-180.
[28] WANG D, SONG C, BARABASI A L. Quantifying long-term scientific impact[J]. Science, 2013, 342(6154):127-132.
[29] TAHAMTAN I, AFSHAR A S, AHAMDZADEH K. Factors affecting number of citations:a comprehensive review of the literature[J]. Scientometrics, 2016, 107(3):1195-1225.
[30] BAI X, ZHANG F, LEE I. Predicting the citations of scholarly paper[J]. Journal of informetrics, 2019, 13(1):407-418.
[31] KOSTEAS V D. Predicting long-run citation counts for articles in top economics journals[J]. Scientometrics, 2018, 115(3):1395-1412.
[32] THELWALL M, NEVILL T. Could scientists use Altmetric.com scores to predict longer term citation counts? [J]. Journal of informetrics, 2018, 12(1):237-248.
[33] WANG M, WANG Z, CHEN G. Which can better predict the future success of articles? Bibliometric indices or alternative metrics[J]. Scientometrics, 2019, 119(3):1575-1595.
[34] MOED H F, DE BRUIN R E, VAN LEEUWEN T N. New bibliometric tools for the assessment of national research performance:database description, overview of indicators and first applications[J]. Scientometrics, 1995, 33(3):381-422.
[35] CWTS. CWTS Leiden Ranking 2019 [EB/OL]. [2021-05-04]. https://www.leidenranking.com/ranking/2019/list.
[36] 胡泽文, 武夷山, 袁军鹏. 典型的高校排名做法与借鉴意义[J]. 高教发展与评估, 2016, 32(2):27-37, 99.
