INFORMATION RESEARCH

The Relationship Between Absolute Disruption Index, Peer Review Index and CNCI: A Study Based on Virology Papers

  • Jiang Yuyan ,
  • Liu Xueli
Expand
  • 1 Henan Research Center for Science Journals, Xinxiang Medical University, Xinxiang 453003;
    2 Periodicals Publishing House, Xinxiang Medical University, Xinxiang 453003

Received date: 2022-08-05

  Revised date: 2022-11-02

  Online published: 2023-02-24

Abstract

[Purpose/Significance] This paper explores the correlation between absolute disruption index (DZ), peer-review index of Faculty Opinions and citation index CNCI and reveals the effect of Faculty Opinions peer review index in early identification of disruption innovation in research papers.[Method/Process] Through the correlation analysis of the selected 140 research papers' DZ, Faculty Opinions peer review indicators[including peer rating (FScore), peer rating (FStar), evaluation times (FTime), weighted rating (FStar_w), weighted evaluation times (FTime_w)], and the impact index CNCI, furthermore, the distribution of high-disruptive literature, literature collected by Faculty Opinions and high-impact literature in all 5 566 focus literatures and the coincidence of research literatures selected from different evaluation angles were studied.[Result/Conclusion] Across the all virology journals, there is a weak correlation between DZ and the peer review index of Faculty Opinions and a moderate correlation between DZ and CNCI. There is a strong correlation between FScore and CNCI, a moderate correlation between FStar and FStar_w and CNCI, and a weak correlation between FTime and FTime_w. But there are different correlations among the three indexes of research papers with different labels. Among them, the consistency and FStar_w between peer review results of transformative research papers and absolute disruption index are higher than that of evidence-based research papers. Among them, the peer review results of transformative research papers are more consistent with absolute subversive index than evidence-based research papers, and average FStar_w of transformative research papers are higher than those of evidence-based research papers. In the actual evaluation effect, the three evaluation indexes should complement each other rather than replace each other. In the process of identifying early disruptive innovations, the peer review index of the Faculty Opinions can play a certain role and it can also assist researchers to quickly find potentially influential research papers after publication at the same time.

Cite this article

Jiang Yuyan , Liu Xueli . The Relationship Between Absolute Disruption Index, Peer Review Index and CNCI: A Study Based on Virology Papers[J]. Library and Information Service, 2023 , 67(3) : 96 -105 . DOI: 10.13266/j.issn.0252-3116.2023.03.009

References

[1] KOSTOFF R N, BOYLAN R, SIMONS G R.Disruptive technology roadmaps[J].Technological forecasting and social change, 2004, 71(1):141-159.
[2] FUNK R J, OWEN-SMITH J.A dynamic network measure of technological change[J].Management science, 2017, 63(3):791-817.
[3] WU L, WANG D, EVANS J A.Large teams develop and small teams disrupt science and technology[J].Nature, 2019, 566(7744):378-382.
[4] BORNMANN L, TEKLES A.Disruptive papers published in Scientometrics[J].Scientometrics, 2019, 120(1):331-336.
[5] BORNMANN L, DEVARAKONDA S, TEKLES A, et al.Are disruption index indicators convergently valid? the comparison of several indicator variants with assessments by peers[J].Quantitative science studies, 2020, 1(3):1242-1259.
[6] RUAN X, LYU D, GONG K, et al.Rethinking the disruption index as a measure of scientific and technological advances[J].Technological forecasting and social change, 2021, 172:121071.
[7] MIN C, BU Y, SUN J.Predicting scientific breakthroughs based on knowledge structure variations[J].Technological forecasting and social change, 2021, 164:120502.
[8] 刘小慧, 沈哲思, 廖宇, 等.颠覆性指数的稳定时间窗研究[J].图书情报工作, 2021, 65(18):49-57.
[9] 刘小慧, 沈哲思, 廖宇, 等.科研论文颠覆性指数的改进及其影响因素研究[J].图书情报工作, 2020, 64(24):84-91.
[10] 刘小慧, 廖宇, 朱曼曼.颠覆性指数用于科研评价初探[J].情报理论与实践, 2021, 44(12):34-40.
[11] 谢岩岩, 孙继林.F1000的文献评价功能浅议[J].图书馆杂志, 2011, 30(4):38-42, 47.
[12] 许咏丽, 孙继林.国际生物医学领域学术评估新利器——Faculty of 1000[J].图书馆, 2011(5):61-63.
[13] 刘春丽.基于软同行评议的科学论文影响力评价方法——F1000因子[J].中国科技期刊研究, 2012, 23(3):383-386.
[14] 宋丽萍, 王建芳.基于F1000与 WoS的同行评议与文献计量相关性研究[J].中国图书馆学报, 2012, 38(2):62-69.
[15] BORNMANN L, LEYDESDORFF L.The validation of (advanced) bibliometric indicators through peer assessments:a comparative study using data from incites and F1000[J].Journal of informetrics, 2013, 7(2):286-291.
[16] WALTMAN L, COSTAS R.F1000 recommendations as a potential new data source for research evaluation:a comparison with citations[J].Journal of the Association for Information Science and Technology, 2014, 65(3):433-445.
[17] 檀旦.F1000与传统文献计量学指标的相关性研究[J].中国科技期刊研究, 2016, 27(1):111-115.
[18] MOHAMMADI E, THELWALL M.Assessing non-standard article impact using F1000 labels[J].Scientometrics, 2013, 97(2):383-395.
[19] BORNMANN L.Validity of altmetrics data for measuring societal impact:a study using data from Altmetric and F1000Prime[J].Journal of informetrics, 2014, 8(4):935-950.
[20] 彭秋茹, 阎素兰, 黄水清.基于全文本分析的引文指标研究——以F1000推荐论文为例[J].信息资源管理学报, 2019, 9(4):82-88.
[21] BORNMANN L.Usefulness of altmetrics for measuring the broader impact of research:a case study using data from PLOS and F1000Prime[J].Aslib journal of information management, 2015, 67(3):305-319.
[22] 曹丽江, 周毅.基于元分析的Altmetrics指标与传统引用指标相关性研究[J].情报理论与实践, 2016, 39(8):49-53.
[23] 陈小清, 刘丽, 邢美园.单篇论著影响力评价指标比较分析——学术迹与Altmetrics评分、F1000评分、Comment的比较[J].情报理论与实践, 2017, 40(3):114-118.
[24] 谭贝加.被引频次结合Altmetrics得分、F1000评分用于生物医学论文影响力评价的可行性研究——以2014-2017年 Altmetrics Top100论文为例[J].中国科技期刊研究, 2020, 31(11):1388-1393.
[25] 许丹, 徐爽, 陈斯斯, 等.基于自然语言词对法的文献主题新颖性探测研究[J].图书情报工作, 2018, 62(8):130-138.
[26] 王雪, 杨雪梅, 林紫洛, 等.基于引文全文本的医学领域突破性文献识别研究[J].情报杂志, 2021, 40(3):132-138.
[27] BORNMANN L, TEKLES A.Disruption index depends on length of citation window[J].El profesional de la información, 2019, 28(2).
[28] PERNER A, HAASE N, GUTTORMSEN A B, et al.Hydroxyethyl starch 130/0.42 versus Ringer's acetate in severe sepsis[J].New England journal of medicine, 2012, 367(2):124-134.
[29] BERG B V D, CLEMONS W M, COLLINSON I, et al.X-ray structure of a protein-conducting channel[J].Nature, 2004, 427(6969):36-44.
[30] RINGACH D L, JENTSCH J D.We must face the threats[J].Journal of neuroscience, 2009, 29(37):11417-11418.
[31] JUMPER J, EVANS R, PRITZEL A, et al.Highly accurate protein structure prediction with AlphaFold[J].Nature, 2021, 596(7873):583-589.
[32] LOLLE S J, VICTOR J L, YOUNG J M, et al.Genome-wide non-mendelian inheritance of extra-genomic information in Arabidopsis[J].Nature, 2005, 434(7032):505-509.
[33] BINSHTOK A M, BEAN B P, WOOLF C J.Inhibition of nociceptors by TRPV1-mediated entry of impermeant sodium channel blockers[J].Nature, 2007, 449(7162):607-610.
[34] PERSSON E K, VERSTRAETE K, HEYNDRICKX I, et al.Protein crystallization promotes type 2 immunity and is reversible by antibody treatment[J].Science, 2019, 364(6442):eaaw4295.
[35] AYOUB K, NAIROOZ R, ALMOMANI A, et al.Perioperative heparin bridging in atrial fibrillation patients requiring temporary interruption of anticoagulation:evidence from meta-analysis[J].Journal of stroke and cerebrovascular diseases, 2016, 25(9):2215-2221.
[36] KRAGL M, KNAPP D, NACU E, et al.Cells keep a memory of their tissue origin during axolotl limb regeneration[J].Nature, 2009, 460(7251):60-65.
[37] JUMPER J, EVANS R, PRITZEL A, et al.Highly accurate protein structure prediction with AlphaFold[J].Nature, 2021, 596(7873):583-589.
[38] DU J, TANG X, WU Y.The effects of research level and article type on the differences between citation metrics and F1000 recommendations[J].Journal of the Association for Information Science and Technology, 2016, 67(12):3008-3021.
[39] BORNMANN L, MUTZ R.Further steps towards an ideal method of measuring citation performance:the avoidance of citation (ratio) averages in field-normalization[J].Journal of informetrics, 2011, 5(1):228-230.
[40] 宋丽萍, 王建芳, 王树义.科学评价视角下F1000、Mendeley与传统文献计量指标的比较[J].中国图书馆学报, 2014, 40(4):48-54.
[41] SILER K, LEE K, BERO L.Measuring the effectiveness of scientific gatekeeping[J].Proceedings of the National Academy of Sciences, 2015, 112(2):360-365.
[42] BREZIS E S, BIRUKOU A.Arbitrariness in the peer review process[J].Scientometrics, 2020, 123(1):393-411.
[43] MARKS M S, MARSH M C P, SCHROER T A, et al.Editorial[J].Traffic, 2013, 14(1):1.
[44] BREZIS E S, BIRUKOU A.Arbitrariness in the peer review process[J].Scientometrics, 2020, 123(1):393-411.
[45] 姜育彦, 刘雪立.开放型同行评议:模式、技术、问题与对策[J].中国科技期刊研究, 2022, 33(9):1196-1205.
Outlines

/