[Purpose/significance] To investigate field-normalized reference influence indicators on the level of papers, the effectiveness of a citation evaluation method (Relative Citation Ratio, RCR) based on the research field identification by co-citation network is analyzed.[Method/process] In order to test the effectiveness of RCR in the evaluation of academic impact, the paper chose iCite, F1000, InCites, then took 739 604 articles as a sample, and analyzed the correlation among RCR, Category Normalized Citation Impact (CNCI) and journal impact factor (JIF) by the R language.[Result/conclusion] The paper shows that RCR strongly correlates with CNCI, and poorly correlates with journal impact factor, which indicate the similarities and differences between RCR and CNCI and confirm that it is a good practice for implementing San Francisco Declaration on Research Assessment and assessing research on its own merits rather than on the basis of the journal in which the research is published. It also turns out the inverse relationship of the correlation and the number of interdisciplinary subjects between RCR and journal impact factor in interdisciplinary science, and lower correlation between RCR and journal impact factor than in a single discipline, which appears the applicability and effectiveness of RCR in evaluating the influence of academic papers in interdisciplinary science. The paper concludes that RCR based on research field identification by co-citation network provides new solution for the evaluation of paper influence, especially for interdisciplinary research outputs.
Song Liping
,
Wang Jianfang
,
Fu Jie
,
Yuan Shanshan
. Analysis on the Effectiveness of Citation Evaluation Methods in the Research Field Identification by Co-Citation Network[J]. Library and Information Service, 2021
, 65(23)
: 100
-105
.
DOI: 10.13266/j.issn.0252-3116.2021.23.011
[1] WALTMAN L. A review of the literature on citation impact indicators[J]. Journal of informetrics, 2016, 10(2):365-391.
[2] GARFIELD E. The history and meaning of the journal impact factor[J]. Journal of the American Medical Association, 2006, 295(1):90-93.
[3] HUTCHINS B I, YUAN X, ANDERSON J M, et al. Relative citation ratio (RCR):a new metric that uses citation rates to measure influence at the article level[J]. PLoS biology, 2016, 14(9):e1002541.
[4] SCHUBERT A, BRAUN T. Relative indicators and relational charts for comparative assessment of publication output and citation impact[J]. Scientometrics, 1986, 9(5/6):281-291.
[5] BORNMANN L, HAUNSCHILD R. Relative citation ratio (RCR):an empirical attempt to study a new field-normalized bibliometric indicator[J]. Journal of the Association for Information Science & Technology, 2017, 68(4):1064-1067.
[6] WALTMAN L,VAN E N J. Field normalization of scientometric indicators[M]//GLANZEL W, MOED H F. Springer handbook of science and technology indicators. Berlin:Springer, 2019:281-300.
[7] Is the relative citation ratio a better metric to evaluate scientific papers?[EB/OL].[2021-08-06]. https://www.enago.com/academy/is-the-relative-citation-ratio-a-better-metric-to-evaluate-scientific-papers/.
[8] PURKAYASTHA A, PALMARO E, FALK-KRZESINSIKI H J, et al. Comparison of two article-level, field-independent citation metrics:field-weighted citation impact (FWCI) and relative citation ratio (RCR)[J]. Journal of informetrics, 2019, 13(2):635-642.
[9] STECK N, STALDER L, EGGER M. Journal- or article-based citation measure? a study of academic promotion at a Swiss university[EB/OL].[2021-08-06].https://f1000research.com/articles/9-1188.
[10] GROGAN D,REDDY V, GUPTA A. Trends in academic spine neurosurgeon productivity as measured by the relative citation ratio[J]. World neurosurg, 2021, 3(147):e40-e46.
[11] SEPPANEN J T, VARRI H, YIONEN I. Co-citation percentile rank and JYUcite:a new network-standardized output-level citation influence metric and its implementation using dimensions API[EB/OL].[2021-08-06]. https://www.biorxiv.org/content/10.1101/2020.09.23.310052v1.
[12] 陈斯斯,刘春丽.基于共引网络的出版物影响力评价新指标:相对引用率[J].情报理论与实践, 2020, 43(7):75-80.
[13] JANSSENS A, GOODMAN M, POWELL K R, et al. A critical evaluation of the algorithm behind the relative citation ratio (RCR)[J]. PLoS biology, 2017, 15(10):e2002536.
[14] EYSENBACH G. Can Tweets predict citations? metrics of social impact based on twitter and correla-tion with traditional metrics of scientific impact[J]. Journal of medical Internet research, 2011, 13(4):e123.
[15] GLANZEL W, SCHUBERT A. A priori vs. a posteriori normalisation of citation indicators. the case of journal ranking[J]. Scientometrics, 2011, 87(2):415-424.
[16] SUGIMOTO C, WEINGART S.The kaleidoscope of disciplinarity[J].Journal of documentation,2015, 71(4):775-794.
[17] ZITT M, RAMANANA-RAHARY S, BASSECOULARD E. Relativity of citation performance and excellence measures:from cross-field to cross-scale effects of field-normalisation[J]. Scientometrics, 2005, 63(2):373-401.
[18] 邱均平, 刘国徽. 国内耦合分析方法研究现状与展望[J]. 图书情报工作, 2014,58(7):137-142.
[19] San Francisco declaration on research assessment[EB/OL].[2021-04-23]. http://www.ascb.org/dora/.
[20] GUÉDON J C,KRAMER B,LAAKSO M,et al. Future of scholarly publishing and scholarly communicate on:report of the expert group to the European commission[EB/OL].[2021-05-08]. https://op.e-ur-opa.e-u/en/publication-detail/-/publication/464477b3-2559-11e9-8d04-01aa75ed71a1.
[21] MALLAPATY S. China bans cash rewards for publishing papers[J]. Nature, 2020, 579(7797):18-19.
[22] ZANOTTOE D, CARVALHO V. Article age-and field-normalized tools to evaluate scientific impact and momentum[J]. Scientometrics, 2021,126(4):2865-2883.
[23] BORNMANN L, MARX W. Critical rationalism and the search for standard (field-normalized) indicators in bibliometrics[J]. Journal of informetrics, 2018, 12(3):598-604.
[24] SCHUBERT A, BRAUN T. Cross-field normalization of scientometric indicators[J]. Scientometrics, 1996, 36(3):311-324.
[25] ALISA S, SPORE S. The relative citation ratio:what is it and why should medical librarians care?[J]. Journal of the Medical Library Association, 2018,106(4):508-513.
