科学计量视角下的论文同行评议研究综述

doi:10.13266/j.issn.0252-3116.2022.14.014

图书情报工作 ›› 2022, Vol. 66 ›› Issue (14): 137-149.DOI: 10.13266/j.issn.0252-3116.2022.14.014

科学计量视角下的论文同行评议研究综述

张光耀^1,2, 谢维熙¹, 姜春林¹, 王贤文¹

1. 大连理工大学科学学与科技管理研究所暨WISE实验室大连 116024;
2. 荷兰马斯特里赫特科技创新经济研究院(UNU-MERIT) 马斯特里赫特 6211AX

收稿日期:2022-02-17 修回日期:2022-05-10 出版日期:2022-07-20 发布日期:2022-07-28
通讯作者: 王贤文,教授,博士,通信作者,E-mail:xianwenwang@dlut.edu.cn。
作者简介:张光耀,博士研究生;谢维熙,博士研究生;姜春林,教授
基金资助:
本文系国家自然科学基金项目"科学文献全景大数据下的研究热点及研究前沿探测"(项目编号:71974029)研究成果之一。

A Review of Peer Review of Papers from the Scientometrics Perspective

Zhang Guangyao^1,2, Xie Weixi¹, Jiang Chunlin¹, Wang Xianwen¹

1. Institute of Science of Science and S&T Management and WISE Lab, Dalian University of Technology, Dalian 116024;
2. Maastricht Economic and Social Research Institute on Innovation and Technology, United Nations University, Maastricht 6211AX

Received:2022-02-17 Revised:2022-05-10 Online:2022-07-20 Published:2022-07-28

摘要/Abstract

摘要： [目的/意义]旨在通过梳理科学计量视角下同行评议相关研究,为深入开展同行评议研究提供参考。[方法/过程]在概括同行评议量化特征的基础上,从同行评议公平性、可靠性、预见性3个方面对国内外研究现状进行综述。[结果/结论]总结科学计量视角下的论文同行评议研究在研究方法多样性和数据应用全面性方面的趋势,并指出当下研究在结论的可解释性上仍然有待提升、国内同行评议实证研究相对不足等问题。

关键词: 同行评议, 开放同行评议, 科学学, 科学计量学

Abstract: [Purpose/significance] By combing related researches of peer review from the scientometrics perspective, this paper provides references for further in-depth research on peer review. [Method/process] This paper reviewed the current foreign and domestic status of research from three aspects:fairness, reliability, and validity of peer review on the basis of generalizing the quantitative characteristics of peer review. [Result/conclusion] The paper summarizes the trends in the diversity of research methods and the comprehensiveness of data application in peer review research from the scientometrics perspective, and points out that the interpretability of the current findings still needs to be improved, and the empirical research on peer review in China is still relatively insufficient.

Key words: peer review, open peer review, science of science, scientometrics

中图分类号:

G250

张光耀, 谢维熙, 姜春林, 王贤文. 科学计量视角下的论文同行评议研究综述[J]. 图书情报工作, 2022, 66(14): 137-149.

Zhang Guangyao, Xie Weixi, Jiang Chunlin, Wang Xianwen. A Review of Peer Review of Papers from the Scientometrics Perspective[J]. Library and Information Service, 2022, 66(14): 137-149.

参考文献

[1] BORNMANN L. Scientific peer review[J]. Annual review of information science and technology, 2011, 45(1):197-245.
[2] MATSUI A, CHEN E, WANG Y, et al. The impact of peer review on the contribution potential of scientific papers[J]. Peerj, 2021, 9:e11999.
[3] FERREIRA C, BASTILLE-ROUSSEAU G, BENNETT A M, et al. The evolution of peer review as a basis for scientific publication:directional selection towards a robust discipline?[J]. Biological reviews, 2016, 91(3):597-610.
[4] GRIMALDO F, MARUIC'A, SQUAZZONI F. Fragments of peer review:a quantitative analysis of the literature (1969-2015)[J]. Plos one, 2018, 13(2):e0193148.
[5] DALTON M S. Refereeing of scholarly works for primary publishing[J]. Annual review of information science and technology, 1995, 30:213-250.
[6] 付伟棠.我国学术期刊同行评议研究综述[J].中国科技期刊研究, 2019, 30(8):819-826.
[7] 常唯,袁境泽.国际学术出版中的同行评议进展与展望[J].中国科技期刊研究, 2020, 31(10):1181-1192.
[8] 索传军,于淼.国外期刊论文同行评议创新态势述评[J].图书情报工作, 2021, 65(1):128-139.
[9] 秦成磊,章成志.大数据环境下同行评议面临的问题与对策[J].情报理论与实践, 2021, 44(4):99-112.
[10] BATAGELJ V, FERLIGOJ A, SQUAZZONI F. The emergence of a field:a network analysis of research on peer review[J]. Scientometrics, 2017, 113(1):503-532.
[11] RENNIE D. Let's make peer review scientific[J]. Nature, 2016, 535(7610):31-33.
[12] LABAND D N. Is there value-added from the review process in economics?:preliminary evidence from authors[J]. Quarterly journal of economics, 1990, 105(2):341-352.
[13] CASNICI N, GRIMALDO F, GILBERT N, et al. Attitudes of referees in a multidisciplinary journal:an empirical analysis[J]. Journal of the Association for Information Science and Technology, 2017, 68(7):1763-1771.
[14] BIANCHI F, GRIMALDO F, SQUAZZONI F. The F-3-index. valuing reviewers for scholarly journals[J]. Journal of informetrics, 2019, 13(1):78-86.
[15] LEE C J, SUGIMOTO C R, ZHANG G, et al. Bias in peer review[J]. Journal of the American Society for Information Science and Technology, 2013, 64(1):2-17.
[16] ZHANG G, XU S, SUN Y, et al. Understanding the peer review endeavor in scientific publishing[J]. Journal of informetrics, 2022, 16(2):101264.
[17] BULJAN I, GARCIA-COSTA D, GRIMALDO F, et al. Large-scale language analysis of peer review reports[J]. Elife, 2020, 9:e53249.
[18] ZHANG L, SHANG Y, HUANG Y, et al. Gender differences among active reviewers:an investigation based on Publons[J]. Scientometrics, 2022, 127(1):145-179.
[19] BRAVO G, GRIMALDO F, LÓPEZ-IÑESTA E, et al. The effect of publishing peer review reports on referee behavior in five scholarly journals[J]. Nature communications, 2019, 10(1):322.
[20] GARCIA-COSTA D, SQUAZZONI F, MEHMANI B, et al. Measuring the developmental function of peer review:a multi-dimensional, cross-disciplinary analysis of peer review reports from 740 academic journals[J]. Ssrn:10.2139/ssrn.3912607, 2021.
[21] BORNMANN L, WOLF M, DANIEL H D. Closed versus open reviewing of journal manuscripts:how far do comments differ in language use?[J]. Scientometrics, 2012, 91(3):843-856.
[22] CHETTY R, SAEZ E, SANDOR L. What policies increase prosocial behavior?an experiment with referees at the journal of public economics[J]. Journal of economic perspectives, 2014, 28(3):169-188.
[23] SALINAS S, MUNCH S B. Where should i send it?optimizing the submission decision process[J]. Plos one, 2015, 10(1):e0115451.
[24] BILALLI B, MUNIR R F, ABELLÓ A. A framework for assessing the peer review duration of journals:case study in computer science[J]. Scientometrics, 2020, 126(1):545-563.
[25] XU S, AN M, AN X. Do scientific publications by editorial board members have shorter publication delays and then higher influence?[J]. Scientometrics, 2021, 126(8):6697-6713.
[26] SARIGOEL E, GARCIA D, SCHOLTES I, et al. Quantifying the effect of editor-author relations on manuscript handling times[J]. Scientometrics, 2017, 113(1):609-631.
[27] MROWINSKI M J, FRONCZAK A, FRONCZAK P, et al. Review time in peer review:quantitative analysis and modelling of editorial workflows[J]. Scientometrics, 2016, 107(1):271-286.
[28] DAVO M del C, VIVES C, ÁLVAREZ-DARDET C. Why are women underused in the JECH peer review process?[J]. Journal of epidemiology&community health, 2003, 57(12):936-937.
[29] SCHMALING K B, BLUME A W. Gender differences in providing peer review to two behavioural science journals, 2006-2015[J]. Learned publishing, 2017, 30(3):221-225.
[30] HUISMAN J, SMITS J. Duration and quality of the peer review process:the author's perspective[J]. Scientometrics, 2017, 113(1):633-650.
[31] PAUTASSO M, SCHÄFER H. Peer review delay and selectivity in ecology journals[J]. Scientometrics, 2010, 84(2):307-315.
[32] OKIKE K, KOCHER M S, NWACHUKWU B U, et al. The fate of manuscripts rejected by the journal of bone and joint surgery (American volume)[J]. The journal of bone&joint surgery, 2012, 94(17):e130.
[33] DECULLIER E, LHÉRITIER V, CHAPUIS F. Fate of biomedical research protocols and publication bias in France:retrospective cohort study[J]. BMJ, 2005, 331(7507):19.
[34] CHAREN D A, MAHER N A, ZUBIZARRETA N, et al. Evaluation of publication delays in the orthopedic surgery manuscript review process from 2010 to 2015[J]. Scientometrics, 2020, 124(2):1127-1135.
[35] HORBACH S P J M. Pandemic publishing:medical journals strongly speed up their publication process for COVID-19[J]. Quantitative science studies, 2020, 1(3):1056-1067.
[36] DELGADO A F, GARRETSON G, FALK DELGADO A. The language of peer review reports on articles published in the BMJ, 2014-2017:an observational study[J]. Scientometrics, 2019, 120(3):1225-1235.
[37] ZHANG G, WANG L, XIE W, et al. "This article is interesting, however":exploring the language use in peer review comment of articles published in the BMJ[J]. Aslib journal of information management, 2022, 74(3):399-416.
[38] 张明阳,王刚,彭起,等.学术论文公开评审平台数据分析[J].计算机科学, 2021, 48(6):63-70.
[39] HORBACH S P J M. No time for that now!qualitative changes in manuscript peer review during the Covid-19 pandemic[J]. Research evaluation, 2021, 30(3):231-239.
[40] HUA X, NIKOLOV M, BADUGU N, et al. Argument mining for understanding peer reviews[C]//2019 conference of the North American Chapter of the Association for Computational Linguistics:human language technologies. Minneapolis:Association for Computational Linguistics, 2019:2131-2137.
[41] QIN C, ZHANG C. Exploring the distribution regularities of referees'comments in IMRAD structure of academic articles[C]//18th International conference on scientometrics and informetrics conference.Leuven:International Society for Scientometrics and Informetrics, 2021:1527-1528.
[42] MENG M, WANG Y, ZHANG C. Building multi-level aspects of peer reviews for academic articles[C]//18th international conference on scientometrics and informetrics conference.Leuven:International Society for Scientometrics and Informetrics, 2021:1519-1520.
[43] KANG D, AMMAR W, DALVI B, et al. A dataset of peer reviews (PeerRead):collection, insights and NLP applications[C]//Proceedings of the 2018 conference of the North American Chapter of the Association for Computational Linguistics:human language technologies. New Orleans:Association for Computational Linguistics, 2018:1647-1661.
[44] 林原,王凯巧,丁堃,等.学术论文的定性评价定量化研究[J].情报理论与实践, 2021, 44(8):28-34.
[45] RASHIDI K, SOTUDEH H, MIRZABEIGI M, et al. Determining the informativeness of comments:a natural language study of F1000Research open peer review reports[J]. Online information review, 2020, 44(7):1327-1345.
[46] VINCENT-LAMARRE P, LARIVIōRE V. Textual analysis of artificial intelligence manuscripts reveals features associated with peer review outcome[J]. Quantitative science studies, 2021, 2(2):662-677.
[47] JIANG S. Understanding authors'psychological reactions to peer reviews:a text mining approach[J]. Scientometrics, 2021, 126(7):6085-6103.
[48] BORDIGNON F. Self-correction of science:a comparative study of negative citations and post-publication peer review[J]. Scientometrics, 2020, 124(2):1225-1239.
[49] ORTEGA J L. The relationship and incidence of three editorial notices in pubpeer:errata, expressions of concern, and retractions[J]. Learned publishing, 2021, 34(2):164-174.
[50] ORTEGA J L. Classification and analysis of pubpeer comments:how a web journal club is used[J]. Journal of the Association for Information Science and Technology, 2022, 73(5), 655-670.
[51] WOLFRAM D, WANG P, ABUZAHRA F. An exploration of referees'comments published in open peer review journals:the characteristics of review language and the association between review scrutiny and citations[J]. Research evaluation, 2021, 30(3):314-322.
[52] LUO J, FELICIANI T, REINHART M, et al. Analysing sentiments in peer review reports:evidence from two science funding agencies[J]. Quantitative science studies, 2022, 2(4):1271-1295.
[53] 姜春林,张立伟,刘盛博.图书情报学期刊"联锁编委"的社会网络分析[J].情报学报, 2014, 33(5):481-490.
[54] 张丽华,曲建升.期刊编委比非编委论文作者能更早探测出研究前沿吗[J].情报杂志, 2017, 36(8):113-119.
[55] YOUK S, PARK H S. Where and what do they publish?editors'and editorial board members'affiliated institutions and the citation counts of their endogenous publications in the field of communication[J]. Scientometrics, 2019, 120(3):1237-1260.
[56] 王兴.国际学术期刊编委数量与科研产出评价指标的相关性研究——以经济学学科世界984所大学为例[J].重庆大学学报(社会科学版), 2017, 23(1):61-70.
[57] 卢小莉,李晶,吴登生.基于期刊编委指数的科研机构学术影响力评价研究:以地质学为例[J].情报学报, 2018, 37(1):14-24.
[58] XIE Y, WU Q, LI X. Editorial team scholarly index (ETSI):an alternative indicator for evaluating academic journal reputation[J]. Scientometrics, 2019, 120(3):1333-1349.
[59] 李江.认可审稿人的学术贡献[J].图书情报知识, 2018(5):2.
[60] ORTEGA J L. Are peer-review activities related to reviewer bibliometric performance?a scientometric analysis of Publons[J]. Scientometrics, 2017, 112(2):947-962.
[61] 刘丽萍,刘春丽.基于Publons平台的审稿人贡献认可与评价研究[J].中国科技期刊研究, 2020, 31(1):99-107.
[62] 雷燕.英文科技期刊学术影响力与审稿人学术及审稿表现关系研究与启示——Publons医学领域期刊实证分析[J].中国科技期刊研究, 2021, 32(2):206-213.
[63] WEI C, BU Y, KANG L, et al. Directionality of paper reviewing and publishing of a scientist:a Granger causality inference[J]. Data science and informetrics, 2021, 1(1):68-80.
[64] SQUAZZONI F, BRAVO G, FARJAM M, et al. Peer review and gender bias:a study on 145 scholarly journals[J]. Science advances, 2021, 7(2):eabd0299.
[65] FOX C W. Difficulty of recruiting reviewers predicts review scores and editorial decisions at six journals of ecology and evolution[J]. Scientometrics, 2017, 113(1):465-477.
[66] GARCIA J A, RODRIGUEZ-SÁNCHEZ R, FDEZ-VALDIVIA J. The interplay between the reviewer's incentives and the journal's quality standard[J]. Scientometrics, 2021, 126(4):3041-3061.
[67] COPIELLO S. On the money value of peer review[J]. Scientometrics, 2018, 115(1):613-620.
[68] BIANCHI F, SQUAZZONI F. Is three better than one?simulating the effect of reviewer selection and behavior on the quality and efficiency of peer review[C]//2015 winter simulation conference. New York:IEEE, 2015:4081-4089.
[69] KALMUKOV Y. An algorithm for automatic assignment of reviewers to papers[J]. Scientometrics, 2020, 124(3):1811-1850.
[70] CHUGHTAI G R, LEE J, SHAHZADI M, et al. An efficient ontology-based topic-specific article recommendation model for best-fit reviewers[J]. Scientometrics, 2020, 122(1):249-265.
[71] GARCIA J A, RODRIGUEZ-SÁNCHEZ R, FDEZ-VALDIVIA J. The author-reviewer game[J]. Scientometrics, 2020, 124(3):2409-2431.
[72] FELICIANI T, LUO J, MA L, et al. A scoping review of simulation models of peer review[J]. Scientometrics, 2019, 121(1):555-594.
[73] 梁玉成,贾小双.数据驱动下的自主行动者建模[J].贵州师范大学学报(社会科学版), 2016(6):31-34.
[74] SOBKOWICZ P. Innovation suppression and clique evolution in peer-review-based, competitive research funding systems:an agent-based model[J]. Journal of artificial societies and social simulation, 2015, 18(2):13.
[75] RIGHI S, TAKÁCS K. The miracle of peer review and development in science:an agent-based model[J]. Scientometrics, 2017, 113(1):587-607.
[76] BIANCHI F, GRIMALDO F, BRAVO G, et al. The peer review game:an agent-based model of scientists facing resource constraints and institutional pressures[J]. Scientometrics, 2018, 116(3):1401-1420.
[77] SQUAZZONI F, GANDELLI C. Saint Matthew strikes again:an agent-based model of peer review and the scientific community structure[J]. Journal of informetrics, 2012, 6(2):265-275.
[78] BAUTISTA CABOTA J, GRIMALDO F, SQUAZZONI F. When competition is pushed too hard. an agent-based model of strategic behaviour of referees in peer review[C]//Proceedings 27th european conference on modelling and simulation. Nottingham:European council modeling&simulation, 2013.
[79] MARSH H W, BORNMANN L, MUTZ R, et al. Gender effects in the peer reviews of grant proposals:a comprehensive meta-analysis comparing traditional and multilevel approaches[J]. Review of educational research, 2009, 79(3):1290-1326.
[80] PINHO-GOMES A C, VASSALLO A, THOMPSON K, et al. Representation of women among editors in chief of leading medical journals[J]. Jama network open, 2021, 4(9):e2123026.
[81] FOX C W, DUFFY M A, FAIRBAIRN D J, et al. Gender diversity of editorial boards and gender differences in the peer review process at six journals of ecology and evolution[J]. Ecology and evolution, 2019, 9(24):13636-13649.
[82] HELMER M, SCHOTTDORF M, NEEF A, et al. Gender bias in scholarly peer review[J]. Elife, 2017, 6:e21718.
[83] SQUAZZONI F, BRAVO G, GRIMALDO F, et al. Gender gap in journal submissions and peer review during the first wave of the covid-19 pandemic. a study on 2329 Elsevier journals[J]. Plos one, 2021, 16(10):e0257919.
[84] MURRAY D, SILER K, LARIVIÉRE V, et al. Gender and international diversity improves equity in peer review[J]. Biorxiv:10.1101/400515, 2018.
[85] DEMAREST B, FREEMAN G, SUGIMOTO C R. The reviewer in the mirror:examining gendered and ethnicized notions of reciprocity in peer review[J]. Scientometrics, 2014, 101(1):717-735.
[86] FOX C W, PAINE C E T. Gender differences in peer review outcomes and manuscript impact at six journals of ecology and evolution[J]. Ecology and evolution, 2019, 9(6):3599-3619.
[87] CARD D, DELLAVIGNA S, FUNK P, et al. Are referees and editors in economics gender neutral?[J]. The quarterly journal of economics, 2020, 135(1):269-327.
[88] FOX C W, BURNS C S, MEYER J A. Editor and reviewer gender influence the peer review process but not peer review outcomes at an ecology journal[J]. Functional ecology, 2016, 30(1):140-153.
[89] BURNS C S, FOX C W. Language and socioeconomics predict geographic variation in peer review outcomes at an ecology journal[J]. Scientometrics, 2017, 113(2):1113-1127.
[90] ZHU H. Home country bias in academic publishing:a case study of the new England journal of medicine[J]. Learned publishing, 2021, 34(4):578-584.
[91] ZHANG X. Effect of reviewer's origin on peer review:China vs. non-China[J]. Learned publishing, 2012, 25(4):265-270.
[92] CAMPOS-ARCEIZ A, PRIMACK R B, KOH L P. Reviewer recommendations and editors'decisions for a conservation journal:is it just a crapshoot?and do Chinese authors get a fair shot?[J]. Biological conservation, 2015, 186:22-27.
[93] WEI Y, LEI L. Institution bias in the new england journal of medicine?a bibliometric analysis of publications (1997-2016)[J]. Scientometrics, 2018, 117(3):1771-1775.
[94] 郭峰,李欣.编辑部偏爱、关系稿与引用率贴水——来自中国经济学权威期刊的证据[J].经济学(季刊), 2017, 16(4):1237-1260.
[95] BLANK R. The effects of double-blind versus single-blind reviewing-experimental-evidence from the American-economic-review[J]. American economic review, 1991, 81(5):1041-1067.
[96] TOMKINS A, ZHANG M, HEAVLIN W D. Reviewer bias in single-versus double-blind peer review[J]. Proceedings of the National Academy of Sciences of the United States of America, 2017, 114(48):12708-12713.
[97] SEEBER M, BACCHELLI A. Does single blind peer review hinder newcomers?[J]. Scientometrics, 2017, 113(1):567-585.
[98] FOX C W, BURNS C S, MUNCY A D, et al. Author-suggested reviewers:gender differences and influences on the peer review process at an ecology journal[J]. Functional ecology, 2017, 31(1):270-280.
[99] SHOPOVSKI J, BOLEK C, BOLEK M. Characteristics of peer review reports:editor-suggested versus author-suggested reviewers[J]. Science and engineering ethics, 2020, 26(2):709-726.
[100] TEPLITSKIY M, ACUNA D, ELAMRANI-RAOULT A, et al. The sociology of scientific validity:how professional networks shape judgement in peer review[J]. Research policy, 2018, 47(9):1825-1841.
[101] BRAVO G, FARJAM M, GRIMALDO MORENO F, et al. Hidden connections:network effects on editorial decisions in four computer science journals[J]. Journal of informetrics, 2018, 12(1):101-112.
[102] MARSH H W, JAYASINGHE U W, BOND N W. Improving the peer-review process for grant applications:reliability, validity, bias, and generalizability[J]. American psychologist, 2008, 63(3):160-168.
[103] CICCHETTI D V. The reliability of peer review for manuscript and grant submissions:a cross-disciplinary investigation[J]. Behavioral and brain sciences, 1991, 14(1):119-135.
[104] BORNMANN L, MUTZ R, DANIEL H D. A reliability-generalization study of journal peer reviews:a multilevel meta-analysis of inter-rater reliability and its determinants[J]. Plos one, 2010, 5(12):e14331.
[105] 王颖,孔爱英,朱蓓,等.科技期刊审稿标准一致性的影响因素及对策[J].中国科技期刊研究, 2014, 25(12):1470-1472.
[106] ROTHWELL P M, MARTYN C N. Reproducibility of peer review in clinical neuroscience-is agreement between reviewers any greater than would be expected by chance alone?[J]. Brain, 2000, 123(9):1964-1969.
[107] 张光耀,谢维熙,夏鑫璐,等.非共识研究的影响力更高还是更低呢?——基于中文开放同行评议审稿意见的分析[EB/OL].[2022-06-21].http://kns.cnki.net/kcms/detail/11.1762.G3.20220329.1252.002.html
[108] 刘欣,李江,吴金闪.同行评议一致性研究[J].信息资源管理学报, 2022, 11(6):10-16.
[109] 岳名亮,李富山,汤宏波,等.期刊审稿专家一致性评价方法及其有效性验证[J].数据分析与知识发现, 2021, 5(4):115-122.
[110] YUE M, TANG H, LIU F, et al. Consistency index:measuring the performances of scholar journal reviewers[J]. Scientometrics, 2021, 126(8):7183-7195.
[111] KRAVITZ R L, FRANKS P, FELDMAN M D, et al. Editorial peer reviewers'recommendations at a general medical journal:are they reliable and do editors care?[J]. Plos one, 2010, 5(4):e10072.
[112] FYFE A, SQUAZZONI F, TORNY D, et al. Managing the growth of peer review at the royal society journals, 1865-1965[J]. Science technology&human values, 2020, 45(3):405-429.
[113] RESNIK D B, ELMORE S A. Ensuring the quality, fairness, and integrity of journal peer review:a possible role of editors[J]. Science and engineering ethics, 2016, 22(1):169-188.
[114] PETERSEN A M. Megajournal mismanagement:manuscript decision bias and anomalous editor activity at plos one[J]. Journal of informetrics, 2019, 13(4):100974.
[115] PETERS D P, CECI S J. Peer-review practices of psychological journals:the fate of published articles, submitted again[J]. Behavioral and brain sciences, 1982, 5(2):187-195.
[116] FRANCOIS O. Arbitrariness of peer review:a bayesian analysis of the NIPS experiment[J]. Arxiv:1507.06411.2015.
[117] BREZIS E S, BIRUKOU A. Arbitrariness in the peer review process[J]. Scientometrics, 2020, 123(1):393-411.
[118] CORTES C, LAWRENCE N D. Inconsistency in conference peer review:revisiting the 2014 NeurIPS experiment[J]. Arxiv:2109.09774.2021.
[119] SEEBER M. How do journals of different rank instruct peer reviewers?reviewer guidelines in the field of management[J]. Scientometrics, 2020, 122(3):1387-1405.
[120] SAKAI D. Who is peer reviewed?comparing publication patterns of peer-reviewed and non-peer-reviewed papers in Japanese political science[J]. Scientometrics, 2019, 121(1):65-80.
[121] HORBACH S P J M, HALFFMAN W. The ability of different peer review procedures to flag problematic publications[J]. Scientometrics, 2019, 118(1):339-373.
[122] LEEK J T, TAUB M A, PINEDA F J. Cooperation between referees and authors increases peer review accuracy[J]. Plos one, 2011, 6(11):e26895.
[123] SODERBERG C, ERRINGTON T, SCHIAVONE S, et al. Initial evidence of research quality of registered reports compared with the standard publishing model[J]. Nature human behaviour, 2021, 5(8):1-8.
[124] SMITH R. Peer review:a flawed process at the heart of science and journals[J]. Journal of the Royal Society of Medicine, 2006, 99(4):178-182.
[125] CHUBIN D E, HACKETT E J. Peerless science:peer review and U.S. science policy[M]. New York:State University of New York Press, 1990.
[126] ABRAMO G, D'ANGELO C A, REALE E. Peer review versus bibliometrics:which method better predicts the scholarly impact of publications?[J]. Scientometrics, 2019, 121(1):537-554.
[127] SMOLINSKY L, SAGE D S, LERCHER A J, et al. Citations versus expert opinions:citation analysis of featured reviews of the American Mathematical Society[J]. Scientometrics, 2021, 126(5):3853-3870.
[128] DU J, TANG X, WU Y. The effects of research level and article type on the differences between citation metrics and F1000 recommendations[J]. Journal of the Association for Information Science and Technology, 2016, 67(12):3008-3021.
[129] JIRSCHITZKA J, OEBERST A, GÖLLNER R, et al. Inter-rater reliability and validity of peer reviews in an interdisciplinary field[J]. Scientometrics, 2017, 113(2):1059-1092.
[130] 谢维熙,张光耀,王贤文.开放同行评议视角下学术论文同行评议得分与被引频次的关系[J].中国科技期刊研究, 2022, 33(1):113-121.
[131] RIGBY J, COX D, JULIAN K. Journal peer review:a bar or bridge?an analysis of a paper's revision history and turnaround time, and the effect on citation[J]. Scientometrics, 2018, 114(3):1087-1105.
[132] SIKDAR S, MARSILI M, GANGULY N, et al. Influence of reviewer interaction network on long-term citations:a case study of the scientific peer-review system of the journal of high energy physics[C]//2017 ACM/IEEE joint conference on digital libraries. Toronto:IEEE, 2017:1-10.
[133] CASNICI N, GRIMALDO F, GILBERT N, et al. Assessing peer review by gauging the fate of rejected manuscripts:the case of the journal of artificial societies and social simulation[J]. Scientometrics, 2017, 113(1):533-546.
[134] SILER K, LEE K, BERO L. Measuring the effectiveness of scientific gatekeeping[J]. Proceedings of the National Academy of Sciences of the United States of America, 2015, 112(2):360-365.
[135] FU D Y, HUGHEY J J. Releasing a preprint is associated with more attention and citations for the peer-reviewed article[J]. Elife, 2019, 8:e52646.
[136] XU F, OU G, MA T, et al. The consistency of impact of preprints and their journal publications[J]. Journal of informetrics, 2021, 15(2):101153.
[137] 刘瑞明,赵仁杰.匿名审稿制度推动了中国的经济学进步吗?——基于双重差分方法的研究[J].经济学(季刊), 2017, 16(1):173-204.
[138] 刘瑞明,赵仁杰.政府支持、制度变革与学术期刊进步——来自中国"名刊工程"的经验证据[J].经济学(季刊), 2020, 19(2):473-498.
[139] NI J, ZHAO Z, SHAO Y, et al. The influence of opening up peer review on the citations of journal articles[J]. Scientometrics, 2021, 126(12):9393-9404.
[140] ZONG Q, XIE Y, LIANG J. Does open peer review improve citation count?evidence from a propensity score matching analysis of PeerJ[J]. Scientometrics, 2020, 125(1):607-623.
[141] SQUAZZONI F, AHRWEILER P, BARROS T, et al. Unlock ways to share data on peer review[J]. Nature, 2020, 578(7796):512-514.
[142] VAN ROOYEN S, DELAMOTHE T, EVANS S J W. Effect on peer review of telling reviewers that their signed reviews might be posted on the Web:randomised controlled trial[J]. BMJ, 2010, 341:c5729.
[143] BERG J, DICKHAUT J, MCCABE K. Trust, reciprocity, and social-history[J]. Games and economic behavior, 1995, 10(1):122-142.
[144] SQUAZZONI F, BRAVO G, TAKACS K. Does incentive provision increase the quality of peer review?an experimental study[J]. Research policy, 2013, 42(1):287-294.
[145] RODRIGUEZ-SÁNCHEZ R, GARCÍA J A, FDEZ-VALDIVIA J. Editorial decisions with informed and uninformed reviewers[J]. Scientometrics, 2018, 117(1):25-43.
[146] 彭琳,杜杏叶.科技期刊实施开放式同行评议策略研究[J].中国科技期刊研究, 2018, 29(11):1114-1121.
[147] 姜春林,张光耀,郭琪琴.复印报刊资料科技哲学卷文献计量分析[J].自然辩证法研究, 2019, 35(12):69-75.
[148] 姜春林,贾龙川,张光耀.人大复印报刊资料《创新政策与管理》文献计量研究[J].山东科技大学学报(社会科学版), 2020, 22(1):31-39.作者贡献说明:张光耀:提出研究思路,论文撰写和论文修改;谢维熙:文献收集和论文修改;姜春林:论文修改;王贤文:提出研究思路,论文修改与审定。

科学计量视角下的论文同行评议研究综述

A Review of Peer Review of Papers from the Scientometrics Perspective

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

编辑推荐

Metrics

[1]	孟凡, 黄文彬, 李孟阳. 文献老化的编码化表示及分析方法[J]. 图书情报工作, 2022, 66(15): 14-22.
[2]	秦成磊, 韩茹雪, 周昊旻, 仲江涛, 章成志. 同行评审意见类型识别及其在不同被引频次下的分布研究[J]. 图书情报工作, 2022, 66(13): 102-117.
[3]	丁洁兰, 刘细文. 科学合作的多样性研究综述[J]. 图书情报工作, 2022, 66(11): 129-138.
[4]	黄国彬, 刘磊, 陈丽. 注册式研究报告的同行评议机制研究[J]. 图书情报工作, 2021, 65(18): 24-35.
[5]	高嵩, 张智雄, 丁颖. 不为繁华易匠心潜心钻研结硕果——孟连生在文献计量学和引文数据库建设方面的探索与影响[J]. 图书情报工作, 2021, 65(15): 22-29.
[6]	王旭. 中国学术期刊话语权评价理论框架建构研究[J]. 图书情报工作, 2021, 65(12): 83-92.
[7]	刘春丽, 臧东宇, 张连生. 开放同行评议期刊论文的引文与社会关注度优势研究[J]. 图书情报工作, 2021, 65(1): 110-119.
[8]	索传军, 于淼. 国外期刊论文同行评议创新态势述评[J]. 图书情报工作, 2021, 65(1): 128-139.
[9]	撒旭, 王健, 范智萱, 刘建平, 张贵兰, 徐波. 数据期刊同行评议视角下科学数据质量评价指标识别[J]. 图书情报工作, 2020, 64(17): 123-130.
[10]	石燕青, 孙建军. 我国图书情报学科在国际上的研究主题特征分析[J]. 图书情报工作, 2018, 62(7): 66-76.
[11]	杜杏叶, 李贺, 王玲, 刘远颖, 易飞, 徐健, 王传清, 王善军, 刘晶晶. 中国学者对学术论文公开同行评议的接受度研究[J]. 图书情报工作, 2018, 62(2): 73-81.
[12]	宋丽萍, 王建芳. 基于学科规范引文影响力与同行评议相关性的科学评价实证研究[J]. 图书情报工作, 2018, 62(18): 122-128.
[13]	袁国华, 寇晶晶, 张建勇, 韩正琪. 基于开放同行评议的学者影响力评价研究——以F1000为例[J]. 图书情报工作, 2018, 62(13): 37-44.
[14]	杨眉, 范秀凤, 潘卫, 董文军, 董珏, 汤莉华. 高校图书馆辅助人才评估服务方法探索与成效分析[J]. 图书情报工作, 2017, 61(5): 29-34.
[15]	丁洁兰, 刘细文, 杨立英, 翟琰琦, 王燕鹏. 科学计量方法在科技政策研究中应用的实证研究[J]. 图书情报工作, 2017, 61(24): 77-86.