［目的/意义］ 科技文献中的概念是对文献中知识高度凝练的表达,通常以定义句的形式出现在科技文献中。自动从概念定义句中抽取概念,能够进一步挖掘科技文献中蕴含的重要知识。［方法/过程］ 通过分析概念定义句的结构、句式等模式特征,提出以WCL数据集为基础的语料构建方案,并采用BERT+BiLSTM+CRF模型学习概念定义句的模式,从而实现概念短语抽取。［结果/结论］ 结合以往对概念定义句模式特征的研究,创新性地提出一种基于序列标注学习概念定义句的组成模式,从而实现概念短语抽取。通过BERT+BiLSTM+CRF模型,有效学习了概念定义句中的上下文语义、句式结构、组成项分布等模式特征,实现了句子中概念短语的抽取。
[Purpose/Significance] Concepts in scientific and technical literatures are highly condensed expressions of knowledge in the literatures and usually appear in scientific and technical literatures in the form of definition sentences. Automatic extraction of concepts from concept definition sentences is an important research topic in scientific and technical literature mining. [Method/Process] By analyzing the structure, syntax and other pattern features of concept definition sentences, this paper proposed a corpus construction scheme based on WCL dataset, and used BERT+BiLSTM+CRF model to learn concept definition sentence patterns to achieve concept phrase extraction. [Result/Conclusion] Based on the previous study of the characteristics of concept definition sentence pattern, this paper creatively proposes the composition pattern to learn concept definition sentences based on sequence labeling, to realize the extraction of concept phrases. Through the BERT+BiLSTM+CRF model, the pattern features such as contextual semantics, sentence structure and constituent term distribution in concept definition sentences are effectively learned to achieve the extraction of concept phrases in definition sentences.
[1] 王忠义,夏立新,李玉海.基于知识内容的数字图书馆跨学科多粒度知识表示模型构建[J].中国图书馆学报,2019,45(6):50-64.
[2] 化柏林.学术论文中方法知识元的类型与描述规则研究[J].中国图书馆学报,2016,42(1):30-40.
[3] STORRER A, WELLINGHOFF S. Automated detection and annotation of term definitions in German text corpora[C]//LREC. Proceedings of the fifth international conference on language resources and evaluation. Genoa:European Language Resources Association, 2006:2373-2376.
[4] CASTRO-SANCHEZ N, SIDOROV G. Analysis of definitions of verbs in an explanatory dictionary for automatic extraction of actants based on detection of patterns[C]//NLDB. NLDB 2010:Natural language processing and information systems. Cardiff:NLDB, 2010:233-239.
[5] ESPONOSA-ANKE L,SAGGION H. Applying dependency relations to definition extraction[C]//METAIS E, MATHIEU R, TEISSEIRE M, et al. Natural language processing and information systems. New York:Springer, 2014:63-74.
[6] HEARST M. Automatic acquisition of hyponyms from large text corpora[C]//Proceedings of the 14th conference on computational linguistics-Volume 2. Gothenburg:Association for Computational Linguistics, 1992:539-545.
[7] BOELLA G, DI-CARO L. Extracting definitions and hypernym relations relying on syntactic dependencies and support vector machines[C]//Proceedings of the 51st annual meeting of the Association for Computational Linguistics (Volume 2:Short Papers). Sofia:Association for Computational Linguistics, 2013:532-537.
[8] BORG C, ROSNER M, PACE G. Automatic grammar rule extraction and ranking for definitions[C]//Proceedings of the seventh international conference on language resources and evaluation. Valletta:European Language Resources Association, 2010:2577-2584.
[9] JIN Y, MIN-YEN K, JUN-PING N, et al. Mining scientific terms and their definitions:a study of the ACL anthology[C]//Proceedings of the 2013 conference on empirical methods in natural language processing. Washington:Association for Computational Linguistics, 2013:780-790.
[10] LI SL, XU B, CHUNG T. Definition extraction with LSTM recurrent neural networks[C]//Chinese computational linguistics and natural language processing based on naturally annotated big data. Yantai:Computational Linguistics Professional Committee of the Chinese Information Society of China, 2016:177-189.
[11] ESPONOSA-ANKE L, SCHOCHAET S. Syntactically aware neural architectures for definition extraction[C]//Proceeding of the 2018 conference of the North American Chapter of the Association for Computational Linguistics:human language technologies, volume 2(short papers). New Orleans:Association for Computational Linguistics, 2018:378-385.
[12] SHVETS A, WANNER L. Concept extraction using pointer-generator networks[J]. ArXiv,2020,abs/2008.11295.
[13] AMARANDEI S, FLECAN A, IONI G, et al. Key biomedical concepts extraction[C]//Workshop on curative power of medical Data. Romania:EUROLAN, 2017:234-240.
[14] YANG X, BIAN J, HOGAN WR, et al. Clinical concept extraction using transformers[J]. Journal of the American Medical Informatics Association, 2020, 27(12):1935-1942.
[15] NAVIGLI R, VELARDI P. Learning word-class lattices for definition and hypernym extraction[C]//Proceedings of the 48th annual meeting of the Association for Computational Linguistics. Uppsala:ACL, 2010:1318-1327.
[16] DEVLIN J, CHANG MW, LEE K, et al. Bert:Pre-training of deep bidirectional transformers for language understanding[J]. ArXiv,2018,abs/1810.04805.
[17] GRAVES A, SCHMIDHUBER J. Framewise phoneme classification with bidirectional LSTM and other neural network architectures[J]. Neural Network, 2005, 18(5/6):602-610.
[18] LAFFERTY J, MCCALLUM A, PEREIRA F. Conditional random fields:probabilistic models for segmenting and labeling sequence data[C]//Proceedings of the eighteenth international conference on machine learning. San Francisco:Association for Computer Machinery, 2001:282-289.
[19] 毛明毅,吴晨,钟义信,等.加入自注意力机制的BERT命名实体识别模型[J].智能系统学报, 2020, 84(4):146-153.
[20] JAWAHAR G, SAGOT B, SEDDAH D. What does BERT learn about the structure of language?[C]//Proceedings of the 57th annual meeting of the Association for Computational Linguistics. Florence:Association for Computational Linguistics, 2019:3651-3657.
[21] VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[J]. ArXiv,2017,abs/1706.03762,2017.
[22] HOCHREITER S, SCHMIDHUBER J. Long short-term memory[J]. Neural computation, 1997, 9(8):1735-1780.
[23] HUANG Z, WEI X, KAI Y. Bidirectional LSTM-CRF models for sequence tagging[J]. ArXiv,2015,abs/1508.01991.
[24] YAN H, DENG B, LI X, et al. TENER:Adapting transformer encoder for named entity recognition[J]. ArXiv,2019,abs/1911.04474.作者贡献说明:李雪思:负责论文撰写、模型试验;张智雄:负责思路指导,实验整体把控;刘欢:负责论文修改。
