A Review on the Clinical Impact Identification and Prediction of Academic Papers from the Perspective of Scientometrics

  • Liu Chunli ,
  • Chen Shuang
Expand
  • 1 Library of China Medical University, Shenyang 110122;
    2 School of Health Management, China Medical University, Shenyang 110122

Received date: 2023-07-18

  Revised date: 2023-10-31

  Online published: 2024-03-27

Abstract

[Purpose/Significance] This paper reviews the relevant literature on the clinical impact identification and prediction of academic papers from the scientometrics to provide reference for promoting scientific research on the clinical impact evaluation. [Method/Process] With the literatures analysis, this paper defined the relevant concepts of clinical impact evaluation of academic papers, clarified its subject attributes, and expounded its motivation. On this basis, it sorted out the quantitative characteristics, identification, and prediction methods of clinical influence, and listed the tools of “cross-border citation” data acquisition. [Result/Conclusion] In terms of identifying and predicting the clinical impact of papers from the perspective of scientometrics, rich achievements have been made. In the future, multidisciplinary and cross-team experts are needed to conduct in-depth theoretical exploration. Accelerate the construction of data infrastructure for clinical impact evaluation; To promote the deep combination of clinical impact evaluation of academic papers and big language model, and create a new chapter of intelligent retrieval and evaluation of multi-dimensional impact of academic papers.

Cite this article

Liu Chunli , Chen Shuang . A Review on the Clinical Impact Identification and Prediction of Academic Papers from the Perspective of Scientometrics[J]. Library and Information Service, 2024 , 68(5) : 132 -148 . DOI: 10.13266/j.issn.0252-3116.2024.05.013

References

[1] 侯定凯.科研非学术影响力的基本特征和机制研究——对英国"科研卓越框架" 教育研究案例的文本分析[J].高校教育管理, 2022, 16(6):59-68.(HOU D K.Features and mechanisms of non-academic impact of research:a textual analysis of the educational research case studies of the Research Excellence Framework in the UK[J].Journal of higher education management, 2022, 16(6):59-68.)
[2] 王冬梅, 王向宁.英国高校科研评估体系借鉴意义[J].科研管理, 2023, 44(3):187-192.(WANG D M, WANG X N.Reference significance of the scientific research evaluation system in British universities[J].Science research management, 2023, 44(3):187-192.)
[3] 王楠, 张莎.构建以跨学科和社会影响为导向的科研评估框架——基于英国"科研卓越框架" 的分析[J].中国高教研究, 2021(8):71-77.(WANG N, ZANG S.Building an interdisciplinary and socially impact-driven research evaluation framework:analysis based on the Research Excellence Framework in UK[J].China higher education research, 2021(8):71-77.)
[4] 王梅, 颜红丽.英国高校科研影响力评估的改革与启示——基于REF2021的分析[J].外国教育研究, 2021, 48(7):58-72.(WANG M, YANG H L.Reform and enlightenment of evaluation on social impact of science in UK's universities:analysis based on Research Excellence Framework 2021[J].Studies in foreign education, 2021, 48(7):58-72.)
[5] 石雪怡.英国大学科研成果评价探究及其对破除"五唯" 的启示[J].中国高校科技, 2021(6):70-74.(SHI X Y.Research on the evaluation of scientific research achievements of British universities and its enlightenment to breaking the "five only"[J].China university science & technology, 2021(6):70-74.)
[6] 钟雪倩.发达国家高校科研社会影响力评价的经验与启示-以英国、荷兰和澳大利亚为例[D].南昌:江西师范大学, 2021.(ZHONG X Q.The experience and enlightenment of social influence evaluation of scientific research in colleges and universities in developed countries:taking the United Kingdom, the Netherlands and Australia as examples[D].Nanchang:Jiangxi Normal University, 2021.)
[7] 茹宁, 闫广芬.非学术影响评价:英国REF科研影响评估的创新性评析[J].国家教育行政学院学报, 2020(9):24-34.(RU N, YAN G F.Non-academic impact assessment:analysis of the reform and innovation of the REF's research impact assessment in the UK[J].Journal of National Academy of Education, 2020(9):24-34.)
[8] 熊佩萱, 茹宁.科研卓越框架(REF):英国大学科研评估制度及启示[J].教育理论与实践, 2020, 40(15):3-6.(XIONG P X, RU N.Research Excellence Framework (REF):the research evaluation system in UK universities and its enlightenment[J].Theory and practice of education, 2020, 40(15):3-6.)
[9] 王楠, 罗珺文.英国高校科研成果的社会影响力评析——基于REF2014评估中6679份影响力案例的再分析[J].清华大学教育研究, 2019, 40(5):75-81.(WANG N, LUO J W.The impact of scientific research from UK higher education institutions based on a synthetic analysis of the 6679 impact case studies in REF2014[J].Tsinghua journal of education, 2019, 40(5):75-81.)
[10] 范英杰, 徐芳.如何看待研究成果社会影响力评价?[J].科学与社会, 2019, 9(1):129-142.(FAN Y J, XU F.How to understand the evaluation of social impact of science[J].Science and society, 2019, 9(1):129-142.)
[11] 伍莺莺, 张昭.英国科研评价体系探析及其启示[J].科技进步与对策, 2016, 33(12):138-142.(WU Y Y, ZHANG Z.Study on research evaluation system in UK and the enlightenments for China[J].Science & technology progress and policy, 2016, 33(12):138-142.)
[12] 徐芳, 刘文斌, 李晓轩.英国REF科研影响力评价的方法及启示[J].科学学与科学技术管理, 2014, 35(7):9-15.(XU F, LIU W B, LI X X.Research impact evaluation within UK REF and its implications[J].Science of science and management of S.& T., 2014, 35(7):9-15.)
[13] 刘莉.英国大学科研评价改革:从RAE到REF[J].科学学与科学技术管理, 2014, 35(2):39-45.(LIU L.Reform of research evaluation in UK university:from RAE to REF[J].Science of science and management of S.& T., 2014, 35(2):39-45.)
[14] 青蒿素协作研究组.一种新的倍半萜内酯-青蒿素[J].科学通报, 1977, 22(3):142.(Artemisinin Collaborative Study Group.A new sesquiterpene lactone:qinghaosu[J].Scientific bulletin, 1977, 22(3):142.)
[15] SURKIS A, HOGLE J A, DIAZGRANADOS D, et al.Classifying publications from the clinical and translational science award program along the translational research spectrum:a machine learning approach[J].Journal of translational medicine, 2016, 14(1):1-14.
[16] GRANT J, COTTRELL R, CLUZEAU F, et al.Evaluating "payback" on biomedical research from papers cited in clinical guidelines:applied bibliometric study[J].BMJ, 2000, 320(7242):1107-1111.
[17] National Center for Advancing Translational Sciences.Translational science spectrum[EB/OL].[2023-12-14].https://ncats.nih.gov/translation/spectrum.
[18] 百度百科.临床影响[EB/OL].[2023-12-28].https://baike.baidu.com/item/%E4%B8%B4%E5%BA%8A%E5%BD%B1%E5%93%8D?fromModule=lemma_search-box.(Baidu Baike.Clinical impact[EB/OL].[2023-12-28].https://baike.baidu.com/item/%E4%B8%B4%E5%BA%8A%E5%BD%B1%E5%93%8D?fromModule=lemma_search-box.)
[19] VOISIN CONSULTING LIFE SCIENCES.Clinical impact[EB/OL].[2023-12-28].https://voisinconsulting.com/resources/glossary/clinical-impact/#.
[20] INSUVIA.Clinical impact[EB/OL].[2023-12-28].https://insuvia.com/glossary/clinical-impact/.
[21] 钱国正, 李友发, 顾爱远.颅内外动脉异常与超声诊断及其临床价值[M].上海:第二军医大学出版社, 2005.(QIAN G Z, LI Y F, GU A Y.Intracranial and extracranial arterial abnormalities and ultrasound diagnosis and its clinical value[M].Shanghai:Second Military Medical University Press, 2005.)
[22] 罗晓, 史丽敏, 王汝龙.药物临床价值评价[J].药品评价, 2010, 7(4):20-24.(LUO X, SHI L M, WANG R L.Evaluation of the clinical value of drugs[J].Drug evaluation, 2010, 7(4):20-24.)
[23] 刘则渊, 陈悦, 朱晓宇.普赖斯对科学学理论的贡献——纪念科学计量学之父普赖斯逝世30周年[J].科学学研究, 2013, 31(12):1761-1772.(LIU Z Y, CHEN Y, ZHU X Y.D.J.Price's contribution to theory of the science of science:commemorating father of scientometrics D.J.Price's 30th anniversary of his death[J].Studies in science of science, 2013, 31(12):1761-1772.)
[24] MACLEOD M R, MICHIE S, ROBERTS I, et al.Biomedical research:increasing value, reducing waste[J].Lancet, 2014, 383(9912):101-104.
[25] HABERER J E, BOUM Y.Behind-the-scenes investment for equity in global health research[J].The new England journal of medicine, 2023(388):387-390.
[26] NATIONAL INSTITUTES OF HEALTH.Grants & funding[EB/OL].[2023-12-28].https://www.nih.gov/grants-funding.
[27] ABUDU R, OLIVER K, BOAZ A.What funders are doing to assess the impact of their investments in health and biomedical research[J].Health research policy and systems, 2022, 20:88.
[28] ZERHOUNI E A.Translational and clinical science:time for a new vision[J].New england journal of medicine, 2005, 353(15):1621-1623.
[29] 格鲁伯, 约翰逊.美国创新简史:科技如何助推经济增长[M].穆凤良, 译.北京:中信出版集团, 2021.(GRUBER J, JOHNSON S.Jump-starting America:how breakthrough science can revive economic growth and the American dream[M].MU F L, trans.Beijing:CITIC Press Group, 2021.)
[30] 郝红全, 赵英弘, 杨好好, 等.2022年度国家自然科学基金项目申请、评审与资助工作综述[J].中国科学基金, 2023, 37(1):3-6.(HAO H Q, ZHAO Y H, YANG H H, et al.Proposal application, peer review and funding of national natural science foundation of China in 2022:an overview[J].Bulletin of national natural science foundation, 2023, 37(1):3-6.)
[31] LAWLER A.Physicist-turned-politician seeks middle ground[J].Science, 1998, 281:635-636.
[32] CONTOPOULOS-IOANNIDIS D G, NTZANI E, IOANNIDIS J P.Translation of highly promising basic science research into clinical applications[J].The American journal of medicine, 2003, 114(6):477-484.
[33] CONTOPOULOS-IOANNIDIS D G, ALEXIOU G A, GOUVIAS T C, et al.Life cycle of translational research for medical interventions[J].Science, 2008, 321(5894):1298-1299.
[34] NARIN F, ROZEK R P.Bibliometric analysis of US pharmaceutical industry research performance[J].Research policy, 1988, 17(3):139-154.
[35] VAN ECK, N J, WALTMAN, L, VAN RAAN, A F J, KLAUTZ, R J M, et al.Citation analysis may severely underestimate the impact of clinical research as compared to basic research[J].Plos one, 2013, 8(4):e62395.
[36] PASTERKAMP G, HOEFER I, PRAKKEN B.Lost in the citation valley[J].Nature biotechnology, 2016, 34:1016-1018.
[37] BUTLER D.Translational research:crossing the valley of death[J].Nature, 2008, 453:840-842.
[38] KE Q.The citation disadvantage of clinical research[J].Journal of informetrics, 2020, 14(1):100998.
[39] MAHDI S, PABLO D, ANDREW N A.Citation counts:are they good predictors of RAE scores[EB/OL].[2023-12-28].https://core.ac.uk/download/pdf/36022747.pdf.
[40] KING J.A review of bibliometric and other science indicators and their role in research evaluation[J].Journal of information science, 1987, 13(5):261-276.
[41] VAN RAAN T.The use of bibliometric analysis in research performance assessment and monitoring of interdisciplinary scientific developments[J].Journal for technology assessment in theory and practice, 2003, 12(1):20-29.
[42] 肖晓旦, 张士靖.医学文献主题标引[M].北京:高等教育出版社, 2008.(XIAO X D, ZHANG S J.Subject indexing of medical literature[M].Beijing:Higher Education Press, 2008.)
[43] Web of Science.Document types[EB/OL].[2023-12-28].https://images.webofknowledge.com/WOKRS531OR13/help/WOK/hs_document_types.html.
[44] National Library of Medicine.Publication characteristics (publication types) with scope notes 2020 Mesh pubtypes[EB/OL].[2023-12-28].https://www.nlm.nih.gov/mesh/pubtypes.html.
[45] 叶鹰.情报学基础教程[M].3版.北京:科学出版社, 2018.(YE Y.Basic course in information science[M].3rd ed.Beijing:China Science Publishing & Media Ltd., 2018.)
[46] 魏明.小议"零次文献"[J].图书馆学研究, 1987(2):40-42.(WEI M.A brief discussion on "original material"[J].Research on library science, 1987(2):40-42.)
[47] NIH.Definition for derived data[EB/OL].[2023-12-28].https://icite.od.nih.gov/user_guide?page_id=ug_data.
[48] ZHANG C Z, MAYR P, LU W, et al.Extraction and evaluation of knowledge entities from scientific documents[J].The journal of data and information science, 2021, 6(3):5.
[49] DING Y, SONG M, HAN J, et al.Entitymetrics:measuring the impact of entities[J].PLos one, 2013, 8(8):e71416.
[50] CHANG X, ZHENG Q H.Knowledge element extraction for knowledge-based learning resources organization[C]//Proceedings of the 6th international conference on advances in Web-based learning.Edinburgh:ICWL, 2007:102-113.
[51] 沈雪莹, 欧石燕.科学文献知识单元抽取及应用研究:梳理与展望[J].情报理论与实践, 2022, 45(12):195-207.(SHEN X Y, OU S Y.Research on extraction and application of knowledge units in scientific literature:reviews and prospects[J].Information studies:theory & application, 2022, 45(12):195-207.)
[52] DIDEGAH F, BOWMAN T D, HOLMBERG K.On the differences between citations and altmetrics:an investigation of factors driving altmetrics vs.citations for Finnish articles[J].Journal of the Association for Information Science & Technology, 2018, 69(6):832-843.
[53] FRASER N, MOMENI F, MAYR P, et al.The relationship between bioRxiv preprints, citations and altmetrics[J].Quantitative science studies, 2020, 1(2):618-638.
[54] 余厚强, 李龙飞.政策文件替代计量指标影响因素研究[J].情报理论与实践, 2021, 44(7):28-36.(YU H Q, LI L F.Research on influencing factors of policy documents altmetrics indicators[J].Information studies:theory & application, 2021, 44(7):28-36.)
[55] 王海涛, 谭宗颖, 陈挺.论文被引频次影响因素研究[J].科学学研究, 2016, 34(2):171-177.(WANG H T, TAN Z Y, CHEN T.Research on the factors affecting papers' citation frequency[J].Studies in science of science, 2016, 34(2):171-177.)
[56] 牟象禹, 龚凯乐, 谢娟, 等.论文被引频次的影响因素研究——以国内图书情报领域为例[J].图书情报知识, 2018(4):43-52.(MOU X Y, GONG K L, XIE J, et al.Contributing factors of citations:an empirical study of library and information science in China[J].Documentation, information & knowledge, 2018(4):43-52.)
[57] 陈仕吉, 江文森, 康温和, 等.论文被引频次的影响因素研究[J].情报杂志, 2020, 39(5):83-88.(CHEN S J, JIANG W S, KANG W H, et al.The study on influential factors of citation impact[J].Journal of intelligence, 2020, 39(5):83-88.)
[58] 张琳, 杜会英, 尚媛媛, 等.性别视角下国家自然科学基金项目负责人的成果产出与影响研究[J].信息资源管理学报, 2021, 11(3):110-120.(ZHANG L, DU H Y, SHANG Y Y, et al.Gender differences in academic performance and impact:an analysis of the NSFC project principal investigator[J].Journal of information resources management, 2021, 11(3):110-120.)
[59] CARPENTER M, GIBB F, HARRIS M, et al.Bibliometric profiles for British academic institutions:an experiment to develop research output indicators[J].Scientometrics, 1988, 14(3/4):213-233.
[60] MCALLISTER P R, NARIN F.Characterization of the research papers of US medical schools[J].Journal of the American Society for Information Science, 1983, 34(2):123-131.
[61] BORDONS M, ZULUETA M A.Comparison of research team activity in two biomedical fields[J].Scientometrics, 1997, 40(3):423-436.
[62] NARIN F, KEITH S B.The intramural role of the NIH biomedical research institute[J].Federation proceedings, 1978, 37(8):2120-2123.
[63] BUTLER L.Funding Australia's basic biomedical research of 1993 and 1994[J].Medical journal of Australia, 1999(171):629-633.
[64] LEWISON G, DEVEY M E.Bibliometric methods for the evaluation of arthritis research[J].Rheumatology (Oxford, England), 1999, 38(1):13-20.
[65] BORDONS M, GOMEZ I, FERNANDEZ M, et al.Local, domestic, and international scientific collaboration in biomedical research[J].Scientometrics, 1996, 37(2):279-295.
[66] MCMILLAN G S, NARIN F, DEEDS D L.An analysis of the critical role of public science in innovation:the case of biotechnology[J].Research policy, 2000, 29(1):1-8.
[67] LEWISON G, PARAJE G.The classification of biomedical journals by research level[J].Scientometrics, 2004, 60(2):145-157.
[68] CAMBROSIO A, KEATING P, MERCIER S, et al.Mapping the emergence and development of translational cancer research[J].European journal of cancer, 2006, 42(18):3140-3148.
[69] THELWALL M, MAFLAHI N.Guideline references and academic citations as evidence of the clinical value of health research[J].Journal of the Association for Information Science and Technology, 2016, 67(4):960-966.
[70] ANDERSEN J P.Association between quality of clinical practice guidelines and citations given to their references[EB/OL].[2023-12-28].https://doi.org/10.48550/arXiv.1301.5782.
[71] LEWISON G, SULLIVAN R.The impact of cancer research:how publications influence UK cancer clinical guidelines[J].British journal of cancer, 2008, 98(12):1944-1950.
[72] LEWISON G.The references on UK cancer clinical guidelines[C]//Proceedings of 11th international conference of the international society for scientometrics and informetrics.Madrid:CSIC, 2007:489-498.
[73] KRYL D, ALLEN L, DOLBY K, et al.Tracking the impact of research on policy and practice:investigating the feasibility of using citations in clinical guidelines for research evaluation[J].BMJ open, 2012, 2(2):e000897.
[74] ERIKSSON M, BILLHULT A, BILLHULT T, et al.A new database of the references on international clinical practice guidelines:a facility for the evaluation of clinical research[J].Scientometrics, 2020, 122(2):1221-1235.
[75] THELWALL M, KOUSHA K.Are citations from clinical trials evidence of higher impact research? an analysis of clinical trials.gov[J].Scientometrics, 2016, 109(2):1341-1351.
[76] HAEUSSLER C, ASSMUS A.Bridging the gap between invention and innovation:Increasing success rates in publicly and industry-funded clinical trials[J].Research policy, 2021, 50(2):104155.
[77] SORENSON O, FLEMING L.Science and the diffusion of knowledge[J].Research policy, 2004, 33(10):1615-1634.
[78] DU J, LI P, GUO Q, et al.Measuring the knowledge translation and convergence in pharmaceutical innovation by fundingscience-technology-innovation linkages analysis[J].Journal of informetrics, 2019, 13(1):132-148.
[79] 李沛鑫, 杜建, 郭倩影, 等.基于专利和论文的药物关联分析[J].中华医学图书情报杂志, 2019, 28(4):1-9.(LI P X, DU J, GUO Q Y, et al.Patents and papers-based analysis of links between drugs[J].Chinese journal of medical library and information science, 2019, 28(4):1-9.)
[80] 杜建, 唐小利.基于科学-技术-产品关联分析测度新药研发成果转化及其启示[J].医学信息学杂志, 2017, 38(6):59-65.(DU J, TANG X L.The measurement of new drug R & D achievement transformation based on the analysis of the correlation among science, technology and product as well as the enlightenment[J].Journal of medical informatics, 2017, 38(6):59-65.)
[81] 杜建.基于多维深层数据关联的医学知识挖掘研究进展[J].农业图书情报, 2019, 31(3):4-12.(DU J.Biomedical knowledge discovery based on big data linkage analysis[J].Journal of library and information science in agriculture, 2019, 31(3):4-12.)
[82] PLUM ANALYTICS.Clinical citations from Plum analytics do what traditional citations cannot[EB/OL].[2023-12-28].https://plumanalytics.com/press/clinical-citations-plum-analyticstraditional-citations-cannot/.
[83] ALTMETRIC.Our sources[EB/OL].[2023-12-28].https://www.altmetric.com/about-us/our-data/our-sources/.
[84] HUTCHINS B I, DAVIS M T, MESEROLL R A, et al.Predicting translational progress in biomedical research[J].PLoS biology, 2019, 17(10):e3000416.
[85] LENS.ORG.Whant is the lens[EB/OL].[2023-12-28].https://about.lens.org/.
[86] Clarivate.Derwent innovations index[EB/OL].[2023-12-28].https://clarivate.com/products/scientific-and-academic-research/research-discovery-and-workflow-solutions/webofscienceplatform/derwent-innovations-index-on-web-of-science/.
[87] WEBER G M.Identifying translational science within the triangle of biomedicine[J].Journal of translational medicine, 2013, 11(1):1-10.
[88] ROSAS S R, SCHOUTEN J T, COPE M T, et al.Modeling the dissemination and uptake of clinical trials results[J].Research evaluation, 2013, 22(3):179-186.
[89] HAN X, WILLIAMS S R, ZUCKERMAN B L.A snapshot of translational research funded by the National Institutes of Health (NIH):a case study using behavioral and social science research awards and Clinical and Translational Science Awards funded publications[J].Plos one, 2018, 13(5):e0196545.
[90] KIM Y H, LEVINE A D, NEHL E J, et al.A bibliometric measure of translational science[J].Scientometrics, 2020, 125(3):2349-2382.
[91] ZANG D Y, LIU C L.Exploring the clinical translation intensity of papers published by the world's top scientists in basic medicine[J].Scientometrics, 2023, 128(4):2371-2416.
[92] BOYACK K W, PATEK M, UNGAR L H, et al.Classification of individual articles from all of science by research level[J].Journal of informetrics, 2014, 8(1):1-12.
[93] ÁLVAREZ-BORNSTEIN B, DÍAZ-FAES A A, BORDONS M.What characterises funded biomedical research? evidence from a basic and a clinical domain[J].Scientometrics, 2019, 119(2):805-825.
[94] KE Q.Identifying translational science through embeddings of controlled vocabularies[J].Journal of the American Medical Informatics Association, 2019, 26(6):516-523.
[95] KE Q.The citation disadvantage of clinical research[J].Journal of informetrics, 2020, 14(1):100998.
[96] LI X, TANG X L, CHENG Q K.Predicting the clinical citation count of biomedical papers using multilayer perceptron neural network[J].Journal of informetrics, 2022, 16(4):101333.
[97] NELSON A, GRAY R J, RUFFLE J K, et al.Deep forecasting of translational impact in medical research[J].Patterns (New York, N.Y.), 2022, 3(5):100483.
[98] MAJOR V, SURKIS A, APHINYANAPHONGS Y.Utility of general and specific word embeddings for classifying translational stages of research[J].AMIA annual symposium proceedings, 2018, 2018:1405-1414.
[99] LI X, TANG X L, LU W.Tracking biomedical articles along the translational continuum:a measure based on biomedical knowledge representation[J].Scientometrics, 2023, 128:1295-1319.
[100] 张先恩, 汪卫华, 阎锡蕴, 等.加强基础研究夯实科技自立自强根基[J].中国科学院院刊, 2023, 38(5):755-758.(ZHANG X E, WANG W H, YAN X Y, et al.Strengthen basic research and consolidate foundation for self-reliance and self-improvement in science and technology[J].Bulletin of Chinese Academy of Sciences, 2023, 38(5):755-758.)
Outlines

/