[Purpose/Significance] This paper analyzes the new orientation of responsible research assessment (RRA) for researchers in the context of open science, so as to provide references for China’s scientific research evaluation under the guidance of breaking the “siwei”. [Method/Process] With the deviation between “wei” and “ze” in the traditional evaluation system as the starting point, typical relevant content texts, such as RRA: Global Research Council Conference Report 2021 by Global Research Council and Towards a Reform of the Research Assessment System: Scoping Report by European Commission, were used as the data source, and the keyword extraction and co-occurrence visualization were performed by WordSmith as well. [Result/Conclusion] It shows that the combination of peer’s judgment and responsible metrics for the quality and contribution identification in terms of researchers’ multiple research outputs and activities, has constituted the basic framework and become the criterion of researchers’ RRA under the open science environment. Among these, counterpart peer review and classified evaluation are regarded as the core rules and the necessary ways to break the bad academic driving factors, such as paper only and SCI supremacy, while meta evaluation provides an evidence-based basis for the improvement of researchers’ evaluation quality and optimization of indicators and methods. The resulting conclusions point out that the framework will be conducive to correcting output-based value, promoting the differentiation and diversification of early career researchers (ECR) career paths, and maximizing the function of research ecosystem and facilitating the shift toward inclusive and healthy research and innovation system. Meanwhile, it provides inspiration for the reform of researchers’ evaluation in China.
Song Liping
,
Diao Wenfang
,
Shi Yafang
,
Han Xinyuan
. The New Orientation of Responsible Research Assessment for Researchers in the Context of Open Science[J]. Library and Information Service, 2023
, 67(9)
: 63
-71
.
DOI: 10.13266/j.issn.0252-3116.2023.09.007
[1] UNESCO. UNESCO recommendation on open science[EB/OL].[2022-12-14]. https://en.unesco.org/science-sustainable-future/open-science/recommendation.
[2] G72021 research compact[EB/OL].[2023-01-02]. https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/1001133/G7_2021_Research_Compact__PDF__356KB__2_pages_.pdf.
[3] WILSDON J, ALLEN L, BELFIORE E, et al. The metric tide:report of the independent review of the role of metrics in research assessment and management[M]. London:SAGE, 2015:25-45.
[4] CURRY S, DE R S, HATCH A, et al. The changing role of funders in responsible research assessment:progress, obstacles & the way ahead[R/OL].[2023-01-02]. https://www.researchgate.net/publication/349762528_The_changing_role_of_funders_in_responsible_research_assessment_progress_obstacles_and_the_way_ahead.
[5] OWEN R, MACNAGHTEN P, STILGOE J. Responsible research and innovation:from science in society to science for society, with society[J]. Science & public policy, 2012, 39(6):751-760.
[6] Reimagining academic assessment:stories of innovation and change[EB/OL].[2023-01-02]. https://sfdora.org/dora-casestudies/.
[7] ONI T, SCIARRINO F, ADESSO G, et al. Let researchers try new paths[J]. Nature, 2016, 538:451-453.
[8] MOHER D, BOUTER L, KLEINERT S, et al. The Hong Kong Principles for assessing researchers:fostering research integrity[J]. Plos biology, 2020, 18(7):e3000737.
[9] GLOBAL RESEARCH COUNCIL. Responsible research assessment:Global Research Council conference report 2021[R/OL].[2023-01-02]. https://www.globalresearchcouncil.org/fileadmin/documents/GRC_Publications/GRC_RRA_Conference_Summary_Report.pdf.
[10] O'CARROLL C, ESPOSITO F, RENTIER B, et al. Evaluation of research careers fully acknowledging open science practices-rewards, incentives and/or recognition for researchers practicing open science[R/OL].[2023-01-02]. https://orbi.uliege.be/handle/2268/215460.
[11] Working group for responsible evaluation of a researcher. Good practice in researcher evaluation:recommendation for the responsible evaluation of a researcher in Finland[R/OL].[2023-01-02]. https://avointiede.fi/sites/default/files/2020-03/responsible-evalution.pdf.
[12] Room for everyone's talent towards a new balance in the recognition and rewards of academics[EB/OL].[2023-01-02]. https://www.universiteitenvannederland.nl/recognitionandrewards/wp-content/uploads/2019/11/Position-paper-Room-foreveryone%E2%80%99s-talent.pdf.
[13] FLORIAN N, JOHN P A I, FRANK M, et al. Six principles for assessing scientists for hiring, promotion, and tenure[EB/OL].[2023-01-02]. https://blogs.lse.ac.uk/impactofsocialsciences/2018/06/04/six-principles-for-assessing-scientists-for-hiringpromotion-and-tenure/.
[14] Tools to advance research assessment (TARA) is a project to facilitate the development of new policies and practices for academic career assessment[EB/OL].[2023-01-02]. https://sfdora.org/project-tara.
[15] European Commission. Towards a reform of the research assessment system scoping report[R/OL].[2023-01-02]. https://op.europa.eu/en/publication-detail/-/publication/36ebb96c-50c5-11ec-91ac-01aa75ed71a1/language-en.
[16] 科技部财政部教育部中科院自然科学基金委. 关于开展减轻青年科研人员负担专项行动的通知[EB/OL].[2023- 01-03]. http://www.gov.cn/zhengce/zhengceku/2022-08/08/content_5704610.htm.
[17] 邱均平, 张裕晨, 周子番. 新时代我国科研评价体系重构中必须处理好八大关系[J]. 中国图书馆学报, 2021, 47(1):47-60.
[18] 叶继元. 学术"全评价"体系与中国特色哲学社会科学学术评价体系的构建与完善[J]. 情报资料工作, 2021, 42(3):15-22.
[19] 李晓轩, 徐芳. "四唯"如何破:中国科学院研究所评价的实践和启示[J]. 中国科学院院刊, 2020, 35(12):1431-1438.
[20] 张琳, GUNNAR S. 科学计量与同行评议相结合的科研评价——国际经验与启示[J]. 情报学报, 2020, 39(8):806-816.
[21] HICKS D, WOUTERS P, WALTMAN L, et al. Bibliometrics:the Leiden Manifesto for research metrics[J]. Nature, 2015, 520(7548):429-431.
[22] Research excellence framework 2021[EB/OL].[2023-01-02]. https://www.ref.ac.uk/.
[23] TAHAMTAN L, BORNMANN L. Altmetrics and societal impact measurements:match or mismatch? a literature review[J]. Profesional de la información, 2020, 29(1):1-29.
[24] VSNU KNAW NWO. Strategy evaluation protocol 2021-2027[EB/OL].[2023-01-02]. https://storage.knaw.nl/2022-06/SEP_2021-2027.pdf.
[25] GRANT J. Academic incentives and research impact:developing reward and recognition systems to better people's lives[EB/OL].[2023-01-02]. https://academyhealth.org/sites/default/files/publication/%5Bfield_date%3Acustom%3AY%5D-%5Bfield_date%3Acustom%3Am%5D/academicincentivesresearchimpact_feb2021.pdf.
[26] European Commission. Annual report on the ERC activities and achievements in 2020[EB/OL].[2023-01-03]. https://erc.europa.eu/sites/default/files/2022-09/erc_annual_report_2020.pdf.
[27] San Francisco declaration on research assessment[EB/OL].[2023-01-22]. http://www.ascb.org/dora/.
[28] Changing the narrative:considering common principles for the use of narrative CVs in grant evaluation[EB/OL].[2023- 01-03]. https://sfdora.org/2022/06/06/%ef%bf%bcchangingthe-narrative-considering-common-principles-for-the-use-ofnarrative-cvs-in-grant-evaluation/.
[29] 俞立平, 潘云涛, 武夷山. 元评价对提高科技评价质量的影响研究[J]. 科学学研究, 2012, 30(4):512-516.
[30] Inorms Research Evaluation Group. The SCOPE framework:a five-stage process for evaluating research responsibly[EB/OL].[2023-01-03]. https://inorms.net/wp-content/uploads/2021/11/21655-scope-guide-v9-1636013361_cc-by.pdf.
[31] STERN L. Lord Stern's review of the research excellence framework:call for evidence[EB/OL].[2023-01-03]. https://core.ac.uk/download/pdf/74380480.pdf.
[32] University College London. UCL bibliometrics policy[EB/OL].[2023-01-02]. https://www.ucl.ac.uk/research/strategy-andpolicy/bibliometrics-ucl/ucl-bibliometrics-policy.
[33] Imperial College London. Using InCites responsibly:a guide to interpretation and good practice[EB/OL].[2023-01-03]. https://spiral.imperial.ac.uk/bitstream/10044/1/75946/7/Using%20InCites%20Responsibly.pdf.
[34] 索姆佐尔, 彭德尔伯里. 通过射束图揭示科研人员的引文表现[R/OL].[2023-01-02]. https://discover.clarivate.com/wp_beamplots_CN.
[35] BORNMANN L, MARX W. Distributions instead of single numbers:percentiles and beam plots for the assessment of single researchers[J]. Journal of the American Society for Information Science & Technology, 2014, 65(1):206-208.
[36] WALTMAN L. A review of the literature on citation impact indicators[J]. Journal of informetrics, 2016, 10(2):365-391.
[37] 俞立平, 张矿伟, 蒋长兵. 推进代表作评价存在的问题与对策研究[J]. 情报学报, 2021, 40(4):345-353.
[38] 刘小慧, 沈哲思, 廖宇, 等. 科研论文颠覆性指数的改进及其影响因素研究[J]. 图书情报工作, 2020, 64(24):84-91.
