[目的/意义] 探讨不同学科分类体系在机构科研影响力评价中的差异及对评价结果的影响。[方法/过程] 以Incites数据库为数据来源,选择5种分类体系、8种分类方案。首先对14 955个机构不同分类方案下的学科标准化引文影响力(Category Normalized Citation Impact,CNCI)进行相关性分析,考察不同分类体系下评价结果的整体相似性。然后以国内双一流建设中的36所高校为例,比较和分析不同分类方案下机构CNCI值的变化情况及差异产生的具体原因,研究分类体系对个体机构评价的影响。[结果/结论] 不同学科分类方案下得到的CNCI值相关性显著(最低相关性达到0.85),即不同分类体系得到的整体评价结果具有较高的相似度。但是不同分类体系下的评价结果也存在聚类特征,OECD、ESI、SCADC、CT1相互之间相关系数高、结果更相近,WoS、CT2和CT3评价结果更接近,分类体系的粒度是决定评价结果的重要因素。36所高校在不同的分类体系下评价结果的整体相关性较高,但个别高校CNCI值变化较大,特别是在热点主题上有突出发文的机构。评价结果的巨大差异其根本原因是论文划分到不同类目中,不同类目下的引用基准值不同。在评价过程中更加推荐粒度较细的分类体系,减少热点主题等对引用基准值的影响。
[Purpose/Significance] This paper analyzes the difference of institute impact evalution based on different subject classification systems and their impact on the evaluation results. [Method/Process] The study selected 5 classification systems, and 8 classification schemes from Incites database. Firstly, it conducted correlation analysis on CNCI of 14,955 institutions based on different classification schemes to study the similariy of the evaluation results under different classification systems. Then it selected 36 double first-class universities in China, compared the changes of CNCI values based on different classification schemes, analyzed the specific reasons for the differences, and studied the effect of classification system on the evaluation of individual institutions. [Result/Conclusion] The CNCI values obtained under different subject classification schemes were significantly correlated (the lowest correlation reached 0.85).This indicates that the evaluation results obtained by different classification systems have high similarity. However, the evaluation results under different classification systems also have clustering characteristics. OECD, ESI, SCADC and CT1 have high correlation coefficients and more similar results, and WoS, CT2 and CT3 evaluation results are more similar. The granularity of classification system is an important factor to determine the evaluation result. The overall correlation of the evaluation results of 36 universities under different classification systems is high, but the CNCI values of some universities vary greatly, especially those institutions with prominent publications on hot topics. The fundamental reason for the great difference in the evaluation results is that the same paper is divided into different categories and the citation baselines under different categories are different. In the evaluation process, more fine-grained classification system is recommended to reduce the influence of hot topics on the citation baselines.
[1] LEYDESDORFF L, BORNMANN L, ZHOU P. Construction of a pragmatic base line for journal classifications and maps based on aggregated journal-journal citation relations[J]. Journal of informetrics, 2016,10(4):902-918.
[2] HAUNSCHILD R, SCHIER H, MARX W, et al. Algorithmically generated subject categories based on citation relations: an empirical micro study using papers on overall water splitting[J]. Journal of informetrics, 2018,12(2):436-447.
[3] WANG Q, WALTMAN L. Large-scale analysis of the accuracy of the journal classification systems of Web of Science and Scopus[J]. Journal of informetrics, 2016,10(2):347-364.
[4] ZITT M, LELU A, CADOT M, et al. Bibliometric delineation of scientific fields[M]//GLANZEL W, MOED H F, SCHMOCH U, et al. Springer handbooks. Cham: Springer, 2019:25-68.
[5] 张蕊, 祖文玲, 邱均平, 等. 2021-2022世界一流大学与一流学科评价及结果分析[J]. 评价与管理, 2021,19(4):67-72.
[6] 中国科研国际竞争力研究课题组. 中国基础研究国际竞争力蓝皮书2015[M].北京:科学出版社.2015.
[7] 软科. 软科排名[EB/OL]. [2022-09-23]. https://www.shanghairanking.cn/.
[8] LEYDESDORFF L, BORNMANN L. The operationalization of "fields" as WoS subject categories (WCs) in evaluative bibliometrics: the cases of "library and information science" and "science & technology studies"[J]. Journal of the Association for Information Science and Technology, 2016,67(3):707-714.
[9] 邱均平, 舒非, 卢坚, 等. 面向评价需求的ESI学科分类与我国一流学科类目的匹配研究[J]. 重庆大学学报(社会科学版), 2021,27(1):110-121.
[10] GLANZEL W, SCHUBERT A. A new classification scheme of science fields and subfields designed for scientometric evaluation purposes[J]. Scientometrics, 2003,56(3):357-367.
[11] ARCHAMBAULT É, BEAUCHESNE O H, CARUSO J. Towards a multilingual, comprehensive and open scientific journal ontology[EB/OL]. [2022-09-20]http://www.science-metrix.com/pdf/ Towards a Multilingual Comprehensive and Open.
[12] BOERNER K, KLAVANS R, PATEK M, et al. Design and update of a classification system: the UCSD map of science[J]. Plos one, 2012,7(e394647):1-10.
[13] WALTMAN L, VAN ECK N J. A new methodology for constructing a publication-level classification system of science[J]. Journal of the American Society for Information Science and Technology, 2012,63(12):2378-2392.
[14] KLAVANS R, BOYACK K W. Which type of citation analysis generates the most accurate taxonomy of scientific and technical knowledge?[J]. Journal of the Association for Information Science and Technology, 2017,68(4):984-998.
[15] PERIANES-RODRIGUEZ A, RUIZ-CASTILLO J. A comparison of the Web of Science and publication-level classification systems of science[J]. Journal of informetrics, 2017,11(1):32-45.
[16] 任元秋, 王兴, 郑钦钦. 不同学科分类方案下不同学科标准化方法效果的比较研究[J]. 图书情报工作, 2021,65(3):84-92.
[17] RUIZ-CASTILLO J, WALTMAN L. Field-normalized citation impact indicators using algorithmically constructed classification systems of science[J]. Journal of informetrics, 2015,9(1):102-117.
[18] HAUNSCHILD R, MARX W, FRENCH B, et al. Relationship between field-normalized indicators calculated with different approaches of field-categorization[C]//STI 2018 conference proceedings. Leiden University:Centre for Science and Technology Studies,2018:11-18.
[19] SZOMSZOR M, ADAMS J, PENDLEBURY D A, et al. Data categorization: understanding choices and outcomes[EB/OL].[2022-09-15].https://ccn.unistra.fr/websites/ccn/documentation/Recherche-Data/ISI-GRR-2021-Data-categorization-understanding-choices-and-outcomes.pdf.
[20] SILE L, GUNS R, VANDERMOERE F, et al. Tracing the context in disciplinary classifications: a bibliometric pairwise comparison of five classifications of journals in the social sciences and humanities[J]. Quantitative science studies, 2021,2(1):65-88.
[21] BENSMAN S J, LEYDESDORFF L. Definition and identification of journals as bibliographic and subject entities: librarianship versus ISI journal citation reports methods and their effect on citation measures[J]. Journal of the American Society for Information Science and Technology, 2009,60(6):1097-1117.
[22] PERIANES-RODRIGUEZ A, RUIZ-CASTILLO J. The impact of classification systems in the evaluation of the research performance of the Leiden Ranking universities[J]. Journal of the Association for Information Science and Technology, 2018,69(8):1046-1053.
[23] 美国新闻与世界报道. U.S.News大学排名[EB/OL]. [2022-09-10]. https://www.usnews.com/rankings.
[24] OECD. Frascati Manual 2015[EB/OL]. [2022-08-06]. https://www.oecd.org/sti/frascati-manual-2015-9789264239012-en.htm.
[25] 国务院学位委员会. 学位授予和人才培养学科目录(2018年4月更新)[EB/OL]. [2022-08-06]. http://www.moe.gov.cn/s78/A22/xwb_left/moe_833/201804/t20180419_333655.html.
[26] 国务院学位委员会, 教育部. 研究生教育学科专业目录管理办法[EB/OL]. [2022-09-15]. http://www.gov.cn/zhengce/zhengceku/2022-09/14/5709785/files/612f061295fb42668601c17a6524f753.pdf.
[27] PUDOVKIN A I, GARFIELD E. Algorithmic procedure for finding semantically related journals[J]. Journal of the American Society for Information Science and Technology, 2002,53(13):1113-1119.
