图书情报工作

图书情报工作 >

2024 , Vol. 68 >Issue 3: 117 - 129

DOI: https://doi.org/10.13266/j.issn.0252-3116.2024.03.011

知识组织

有限样本下的科技文献语步识别方法探讨

  • 张鑫 ,
  • 许海云 ,
  • 杨宁 ,
  • 方肖 ,
  • 赵爽
展开
  • 1 中国科学院成都文献情报中心 成都 610213;
    2 中国科学院大学信息资源管理系 北京 100190;
    3 山东理工大学管理学院 淄博 255000
张鑫,馆员,博士研究生,E-mail:zhangxin@clas.ac.cn;许海云,教授,博士,博士生导师;杨宁,副研究馆员,博士;方肖,馆员,博士;赵爽,馆员,硕士。

收稿日期: 2022-12-23

  修回日期: 2023-10-13

  网络出版日期: 2024-02-23

基金资助

本文系四川省社会科学规划项目“专利成果转移转化潜力评价与实证研究”（项目编号：SC22C002）和中国科学院文献情报能力建设专项“‘智慧数据+AI’支撑科学实验操作规程的智能生成研究”（项目编号：E2C0003009）研究成果之一。

收起

Discussion of Moves Recognition of Scientific Documents Under Limited Samples

  • Zhang Xin ,
  • Xu Haiyun ,
  • Yang Ning ,
  • Fang Xiao ,
  • Zhao Shuang
Expand
  • 1 National Science Library (Chengdu), Chinese Academy of Sciences, Chengdu 610213;
    2 Department of Information Resources Management, School of Economics and Management, University of Chinese Academy of Sciences, Beijing 100190;
    3 Business School, Shandong University of Technology, Zibo 255000

Received date: 2022-12-23

  Revised date: 2023-10-13

  Online published: 2024-02-23

Supported by

This work is supported by Social Science Planning of Sichuan Province titled "Evaluation and Empirical Research on the Potential of Patent Achievement Transfer and Transformation" (Grant No. SC22C002), and the Chinese Academy of Sciences Document and Information Capacity Building Project titled "Research on Intelligent Generation of ‘Smart Data+AI’ Supporting Scientific Experiment Operating Procedures" (Grant No. E2C0003009).

Fold

摘要

[目的/意义] 科技文献语步识别是从非结构化的文献中抽取出研究目的、对象、方法、结果、结论等语义片段，针对摘要语步识别实际应用中常出现的高质量标注样本数量较有限、深度识别模型可解释性差等问题开展研究。[方法/过程] 在语步识别中引入提示学习范式，设计对应提示模板和同义词表达器，采用局部线性代理方式生成模型解释，构建可解释的深度学习识别模型，并在生物领域和计算机领域两个数据集随机抽取部分数据中进行模拟实证研究。[结果/结论] 基于大模型提示学习的范式在语步识别任务上以较少训练代价的取得比精调小模型更高的精度，在PubMed三个子数据集上训练后，预测精度分别提高2.5%，4.1%和3.9%。结合准确率和解释结果来看，“方法”“结果”语步识别效果较好（F1值约90%），“背景”“对象”语步相对差些（F1值不到70%）。基于提示学习的方式能够以更快捷高效的方式使用预训练语言模型，获得准确性高、可解释性好的识别模型。

关键词： 小样本; 文本增强; 提示学习; 模型解释

本文引用格式

张鑫 , 许海云 , 杨宁 , 方肖 , 赵爽 . 有限样本下的科技文献语步识别方法探讨[J]. 图书情报工作, 2024 , 68(3) : 117 -129 . DOI: 10.13266/j.issn.0252-3116.2024.03.011

Abstract

[Purpose/Significance] Moves recognition refers to extracting semantic segments such as research purposes, objects, methods, results, and conclusions from unstructured abstracts. This paper focuses on the problems of the limited high-quality annotation samples and the poor interpretability of the deep recognition model that often occur in the practical application of move recognition. [Method/Process] In this paper, it introduced the prompt-based learning paradigm in move recognition, and designed the corresponding template and verbalizer. By the local linear proxy approach, it produced the model interpretation and constructed an interpretable deep learning recognition model. Then it carried out a simulation empirical study on randomly selected partial data from two datasets in the biological and computer fields. [Result/Conclusion] Prompt tuning on large model can achieve higher accuracy than the fine-tuned small model on moves recognition task with less training cost. After training on three sub datasets of PubMed, the F1 score was improved by 2.5%, 4.1% and 3.9%, respectively. Combining the accuracy rate and interpretation results, the “method” and “result” moves recognition effect is better (F1 score about 90%), and the “background” and “method” moves are relatively poor (F1 score <70%). The prompt learning approaches are faster and more efficient to use large pre-trained language model, and obtain recognition results with high accuracy and interpretability.

Key words： limited samples; data augmentation; prompt turning; interpretability

参考文献

[1] SWALES J M. Research genres:explorations and applications[M]. Cambridge:Cambridge University Press, 2004:228-229.
[2] 黄红, 陈翀, 张婧莹. 科技文献内容语义识别研究综述[J]. 情报学报, 2022, 41(9):991-1002. (HUANG H, CHEN C, ZHANG Q. Review on identifying the semantics of scientific[J]. Journal of the China Society for Scientific and Technical Information, 2022, 41(9):991-1002.)
[3] ANTHONY L E. A machine learning system for the automatic identification of text structure, and application to research article abstracts in computer science[D]. Birmingham:University of Birmingham, 2002.
[4] ANTHONY L E, LASHKIA G V. Mover:a machine learning tool to assist in the reading and writing of technical papers[J]. IEEE transactions on professional communication, 2003, 46(3):185-193.
[5] HIROHATA K, OKAZAKI N, ANANIADOU S, et al. Identifying sections in scientific abstracts using conditional random fields[C]//Proceedings of the third international joint conference on natural language processing. Prague:The Association for Computational Linguistics, 2008.
[6] 刘霞. 英语学术论文摘要语步结构自动识别模型的构建[D]. 北京:北京外国语大学, 2016. (LIU X. Constructing a model for the automatic identification of move structure in English research article abstracts[D]. Beijing:Beijing Foreign Studies University, 2016.)
[7] 王立非, 刘霞. 英语学术论文摘要语步结构自动识别模型的构建[J]. 外语电化教学, 2017(2):45-50, 64. (WANG L, LIU X. Constructing a model for the automatic identification of move structure in English research article abstracts[J]. TEFLE, 2017(2):45-50, 64.)
[8] SOONKLANG T. Move classification in scientific abstracts using linguistic features[C]//Proceedings of the eleventh international symposium on natural language processing. Phranakhon Si Ayutthaya:SNLP, 2016.
[9] GUO Y, KORHONEN A, LIAKATA M, et al. Identifying the information structure of scientific abstracts:an investigation of three different schemes[C]//Proceedings of the 2010 workshop on biomedical natural language processing. Uppsala:Association for Computational Linguistics, 2010:99-107.
[10] 杜圣梅, 朱礼军, 徐硕. 面向循证医学的科技文献摘要结构化表示研究[J]. 中国科技资源导刊, 2018, 50(6):94-100. (DU S M, ZHU L J, XU S. Research on structured presentation of scientific literature abstracts for evidence-based medicine[J]. China science & technology resources review, 2018, 50(6):94-100.)
[11] 丁良萍, 张智雄, 刘欢. 影响支持向量机模型语步自动识别效果的因素研究[J]. 数据分析与知识发现, 2019, 3(11):16-23. (DING L P, ZHANG Z X, LIU H. Factors affecting rhetorical move recognition with SVM model[J]. Data analysis and knowledge discovery, 2019, 3(11):16-23.)
[12] 郑梦悦, 秦春秀, 马续补. 面向中文科技文献非结构化摘要的知识元表示与抽取研究——基于知识元本体理论[J]. 情报理论与实践, 2020(2):157-163. (ZHENG M Y, QIN C X, MA X B. Research on knowledge unit representation and extraction for unstructured abstracts of Chinese scientific and technical literature:ontology theory based on knowledge unit[J]. Information studies:theory & application, 2020(2):157-163.)
[13] DAYRELL C, CANDIDO JR A, LIMA G, et al. Rhetorical move detection in English abstracts:multi-label sentence classifiers and their annotated corpora[C]//Proceedings of the 8th international conference on language resources and evaluation. Istanbul:The international conference on language resources and evaluation, 2012:1604-1609.
[14] JIN D, SZOLOVITS P. Hierarchical neural networks for sequential sentence classification in medical scientific abstracts[C]//Proceedings of the conference on empirical methods in natural language processing. Brussels:EMNLP, 2018.
[15] 张智雄, 刘欢, 丁良萍, 等. 不同深度学习模型的科技论文摘要语步识别效果对比研究[J]. 数据分析与知识发现, 2019, 3(12):1-9. (ZHANG Z X, LIU H, DING L P, et al. Identifying moves of research abstracts with deep learning methods[J]. Data analysis and knowledge discovery, 2019, 3(12):1-9.)
[16] 李楠, 方丽, 张逸飞. 学术文本结构功能深度学习识别方法的多学科对比分析[J]. 现代情报, 2019(12):55-63, 87. (LI N, FANG L, ZHANG Y F. Multi-disciplinary comparative study on methods of academic text structure function recognition based on deep learning model[J]. Journal of modern information, 2019(12):55-63, 87.)
[17] DEVLIN J, CHANG M W, LEE K, et al. Bert:Pre-training of deep bidirectional transformers for language understanding[C]//Proceedings of the 2019 conference of the North American Chapter of the Association for Computational Linguistics:human language technologies. Minneapolis:Association for Computational Linguistics, 2019:4171-4186.
[18] ZHANG Z, LIU H, DING L. Moves recognition in abstract of research paper based on deep learning[C]//Proceedings of the 2019 ACM/IEEE joint conference on digital libraries. Illinois:IEEE, 2019.
[19] 刘忠宝, 王宇飞, 张志剑. 基于深度学习模型的摘要结构功能识别方法研究[J]. 情报科学, 2021, 39(3):107-112. (LIU Z B, WANG Y F, ZHANG Z J. Research on the recognition method of abstract structure function based on deep learning model[J]. Information science, 2021, 39(3):107-112.)
[20] 王末, 崔运鹏, 陈丽, 等. 基于深度学习的学术论文语步结构分类方法研究[J]. 数据分析与知识发现, 2020, 4(6):60-68. (WANG M, CUI Y P, CHEN L, et al. A deep learning-based method of argumentative zoning for research articles[J]. Data analysis and knowledge discovery, 2020, 4(6):60-68.)
[21] 欧石燕, 陈嘉文. 科学论文全文语步自动识别研究[J]. 现代情报, 2021, 41(11):3-11. (OU S Y, CHEN J W. The research on automatic recognition of moves in full-text scientific papers[J]. Journal of modern information, 2021, 41(11):3-11.)
[22] 赵旸, 张智雄, 刘欢, 等. 基金项目摘要的语步识别系统设计与实现[J]. 情报理论与实践, 2022, 45(8):162-168. (ZHAO Y, ZHANG Z X, LIU H, et al. Design and implementation of the move recognition system for fund project abstract[J]. Information studies:theory & application, 2022, 45(8):162-168.)
[23] 刘欢, 张智雄, 王宇飞. BERT模型的主要优化改进方法研究综述[J]. 数据分析与知识发现, 2021, 5(1):3-15. (LIU H, ZHANG Z X, WANG Y F. A review on main optimization methods of BERT[J]. Data analysis and knowledge discovery, 2021, 5(1):3-15.)
[24] YU G, ZHANG Z, LIU H, et al. Masked sentence model based on BERT for move recognition in medical scientific abstracts[J]. Journal of data and information science, 2019, 4(4):42-55.
[25] 张智雄, 刘欢, 于改红. 构建基于科技文献知识的人工智能引擎[J]. 农业图书情报学报, 2021, 33(1):17-31. (ZHANG Z X, LIU H, YU G H. Building an artificial intelligence engine based on scientific and technological literature knowledge[J]. Journal of library and information science in agriculture, 2021, 33(1):17-31.)
[26] 郭航程, 何彦青, 兰天, 等. 基于Paragraph-BERT-CRF的科技论文摘要语步功能信息识别方法研究[J]. 数据分析与知识发现, 2022, 6(Z1):298-307. (GUO H C, HE Y Q, LAN T, et al. Identifying moves from scientific abstracts based on ParagraphBERT-CRF[J]. Data analysis and knowledge discovery, 2022, 6(Z1):298-307.)
[27] 刘江峰, 冯钰童, 刘浏, 等. 领域双语数据增强的学术文本摘要结构识别研究[J]. 数据分析与知识发现, 2023, 7(8):105-118. (LIU J F, FENG Y T, LIU L, et al. Structural recognition of abstracts of academic text enhanced by domain bilingual data[J]. Data analysis and knowledge discovery, 2023, 7(8):105-118.)
[28] SUN T, LIU X, QIU X, et al. Paradigm shift in natural language processing[J]. Machine intelligence research, 2021(19):169-183.
[29] DING N, HU S, ZHAO W, et al. OpenPrompt:an opensource framework for prompt-learning[J]. arXiv preprint arXiv:2111.01998, 2021.
[30] DAY R A. The origins of the scientific paper:the IMRAD format[J]. Journal of the American Medical Writers Association, 1989, 4(2):16-18.
[31] SHORTEN C, KHOSHGOFTAAR T M, FURHT B. Text data augmentation for deep learning[J]. Journal of big data, 2021, 8(1):1-34.
[32] WEI J, ZOU K. Eda:easy data augmentation techniques for boosting performance on text classification tasks[J]. arXiv preprint arXiv:1901.11196, 2019.
[33] XIE Q, DAI Z, HOVY E, et al. Unsupervised data augmentation for consistency training[J]. Advances in neural information processing systems, 2020, 33(1):6256-6268.
[34] BROWN T, MANN B, RYDER N, et al. Language models are few-shot learners[J]. Advances in neural information processing systems, 2020, 33(1):1877-1901.
[35] SCHICK T, H SCHÜTZE. Exploiting cloze-questions for fewshot text classification and natural language inference[C]//Proceedings of the conference of the European Chapter of the Association for Computational Linguistics. Bangkok:Association for Computational Linguistics, 2021.
[36] LIU P, YUAN W, FU J, et al. Pre-train, prompt, and predict:a systematic survey of prompting methods in natural language processing[J]. arXiv preprint arXiv:2107.13586, 2021.
[37] RAFFEL C, SHAZEER N, ROBERTS A, et al. Exploring the limits of transfer learning with a unified text-to-text transformer[J]. Journal of machine learning research, 2020, 21(140):1-67.
[38] LESTER B, AL-RFOU R, Constant N. The power of scale for parameter-efficient prompt tuning[J]. arXiv preprint arXiv:2104.08691, 2021.
[39] DING N, QIN Y, YANG G, et al. Delta tuning:a comprehensive study of parameter efficient methods for pre-trained language models[J]. arXiv preprint arXiv:2203.06904, 2022.
[40] YAMAMOTO Y, TAKAGI T. A sentence classification system for multi biomedical literature summarization[C]//21st international conference on data engineering workshops. Tokyo:IEEE, 2005:1163-1163.
[41] RIBEIRO M T, SINGH S, GUESTRIN C. "Why should I trust you?":explaining the predictions of any classifier[C]//Proceedings of the 22nd ACM SIGKDD international conference. New York:ACM, 2016.
[42] LEE C H. A gradient approach for value weighted classification learning in Naive Bayes[J]. Knowledge-based systems, 2015, 85(9):71-79.
[43] BASU A, WALTERS C, SHEPHERD M. Support vector machines for text categorization[C]//36th annual Hawaii international conference on system sciences. Big Island:IEEE, 2003.
[44] WOLF T, DEBUT L, SANH V, et al. Transformers:state-ofthe-art natural language processing[C]//Proceedings of the 2020 conference on empirical methods in natural language processing:system demonstrations. Seattle:Association for Computational Linguistics, 2020.
[45] MORRIS J X, LIFLAND E, YOO J Y, et al. Text attack:a framework for adversarial attacks, data augmentation, and adversarial training in NLP[J]. arXiv preprint arXiv:2005.05909, 2020.
[46] KAPLAN J, MCCANDLISH S, HENIGHAN T, et al. Scaling laws for neural language models[J]. arXiv preprint arXiv:2001.08361, 2020.
[47] WEI J, TAY Y, BOMMASANI R, et al. Emergent abilities of large language models[J]. arXiv preprint arXiv:2206.07682, 2022.
Options
文章导航

/

〈 〉