融合语义联想和BERT的图情领域SAO短文本分类研究

doi:10.13266/j.issn.0252-3116.2021.16.013

图书情报工作 ›› 2021, Vol. 65 ›› Issue (16): 118-129.DOI: 10.13266/j.issn.0252-3116.2021.16.013

融合语义联想和BERT的图情领域SAO短文本分类研究

张玉洁¹, 白如江¹, 刘明月¹, 于纯良²

1 山东理工大学信息管理研究院淄博 255049;
2 烟台大学图书馆烟台 264005

收稿日期:2021-01-27 修回日期:2021-05-12 出版日期:2021-08-20 发布日期:2021-08-20
通讯作者: 白如江(ORCID:0000-0003-3822-8484),研究馆员,硕士生导师,通讯作者,E-mail:brj@sdut.edu.cn。
作者简介:张玉洁(ORCID:0000-0002-6819-031X),硕士研究生;刘明月(ORCID:0000-0002-4335-9369),硕士研究生;于纯良(ORCID:0000-0002-3013-8022),副研究馆员。
基金资助:
本文系山东省高等学校青创科技支持计划"科技大数据驱动的智慧决策支持创新团队-面向新旧动能转换的新兴科学研究前沿识别研究"（项目编号：2019RWG033）和山东省社科规划处项目"数字环境下科学论文的内容标注模型研究"（项目编号：20CSDJ65）研究成果之一。

Research on SAO Short Text Classification in LIS Based on Semantic Association and BERT

Zhang Yujie¹, Bai Rujiang¹, Liu Mingyue¹, Yu Chunliang²

1 Institute of Information Management, Shandong University of Technology, Zibo 255049;
2 Yantai University Library, Yantai 264005

Received:2021-01-27 Revised:2021-05-12 Online:2021-08-20 Published:2021-08-20

摘要/Abstract

摘要： [目的/意义] 针对SAO结构短文本分类时面临的语义特征短缺和领域知识不足问题，提出一种融合语义联想和BERT的SAO分类方法，以期提高短文本分类效果。[方法/过程] 以图情领域SAO短文本为数据源，首先设计了一种包含"扩展-重构-降噪"三环节的语义联想方案，即通过语义扩展和SAO重构延展SAO语义信息，通过语义降噪解决扩展后的噪声干扰问题；然后利用BERT模型对语义联想后的SAO短文本进行训练；最后在分类部分实现自动分类。[结果/结论] 在分别对比了不同联想值、学习率和分类器后，实验结果表明当联想值为10、学习率为4e-5时SAO短文本分类效果达到最优，平均F1值为0.852 2，与SVM、LSTM和单纯的BERT相比，F1值分别提高了0.103 1、0.153 8和0.140 5。

关键词: SAO, 短文本分类, 语义联想, BERT

Abstract: [Purpose/significance] Aiming at the shortage of semantic features and insufficient domain knowledge in the classification of SAO structure short texts, this paper proposes a SAO classification method combining semantic association and BERT in order to improve the classification effect.[Method/process] Taking the SAO short text in the library and information science field as the data source, firstly, a semantic association scheme including the three links of "Expansion-Reconstruction-NoiseReduction" was designed. The semantic information of SAO was extended through semantic expansion and SAO reconstruction, and the extended noise interference problem was solved by semantic noise reduction; then used the BERT model to train the SAO short text after semantic association; finally realized automatic classification in the classification part.[Result/conclusion] After comparing different association values, learning rates and classifiers, the experimental results show that when the association value is 10 and the learning rate is 4e-5, the SAO short text classification effect is optimal, and the average F1 value is 0.852 2, which is comparable to SVM and LSTM compared with pure BERT, the F1 value is increased by 0.103 1, 0.153 8 and 0.140 5 respectively.

Key words: SAO, short text classification, semantic association, BERT

中图分类号:

TP391G250

张玉洁, 白如江, 刘明月, 于纯良. 融合语义联想和BERT的图情领域SAO短文本分类研究[J]. 图书情报工作, 2021, 65(16): 118-129.

Zhang Yujie, Bai Rujiang, Liu Mingyue, Yu Chunliang. Research on SAO Short Text Classification in LIS Based on Semantic Association and BERT[J]. Library and Information Service, 2021, 65(16): 118-129.

参考文献

[1] CASCINI G, FANTECHI A, SPINICCI E. Natural language processing of patents and technical documentation[C]//International workshop on document analysis systems. Berlin:Springer, 2004:508-520.
[2] CHOI S, PARK H, KANG D, et al. An sao-based text mining approach to building a technology tree for technology planning[J]. Expert systems with applications, 2012, 39(13):11443-11455.
[3] WANG X, WANG Z, HUANG Y, et al. Identifying r&d partners through subject-action-object semantic analysis in a problem & solution pattern[J]. Technology analysis & strategic management, 2017, 29(10):1167-1180.
[4] TSOURIKOV V M, BATCHILO L S, SOVPEL I V. Document semantic analysis/selection with knowledge creativity capability utilizing subject-action-object (sao) structures:U.S. Patent 6,167,370[P]. 2000-12-26.
[5] 付芸,汪雪锋,李佳,等.基于SAO结构的创新解决方案遴选研究——以空气净化技术为例[J].图书情报工作,2019,63(6):75-84.
[6] 许海云,王振蒙,胡正银,等.利用专利文本分析识别技术主题的关键技术研究综述[J].情报理论与实践,2016,39(11):131-137.
[7] 胡正银,刘春江,隗玲,等,文奕.面向TRIZ的领域专利技术挖掘系统设计与实践[J].图书情报工作,2017,61(1):117-124.
[8] 杨超,朱东华,汪雪锋,等.专利技术主题分析:基于SAO结构的LDA主题模型方法[J].图书情报工作,2017,61(3):86-96.
[9] CHANG P L, WU C C, Leu H J. Using patent analyses to monitor the technological trends in an emerging field of technology:a case of carbon nanotube field emission display[J]. Scientometrics, 2010, 82(1):5-19.
[10] GUO J, WANG X, LI Q, et al. Subject-action-object-based morphology analysis for determining the direction of technological change[J]. Technological forecasting and social change, 2016, 105:27-40.
[11] LI X, WANG J J, YANG Z. Identifying emerging technologies based on subject-action-object[J]. Journal of intelligence, 2016, 35(3):80-84.
[12] 王晓宇,苗红,王芳.技术知识的跨领域应用及潜在技术方案的识别[J].图书情报工作,2016,60(23):87-96.
[13] 胡正银,方曙,张娴,等.个性化语义TRIZ构建研究[J].图书情报工作,2015,59(7):123-131.
[14] MANEK A S, SHENOY P D, MOHAN M C, et al. Aspect term extraction for sentiment analysis in large movie reviews using gini index feature selection method and SVM classifier[J]. World Wide Web,2017, 20(2):135-154.
[15] BACHHETY S, DHINGRA S, JAIN R, et al. Improved multinomial naïve bayes approach for sentiment analysis on social media[J]. International journal of information systems & management science, 2018, 1(1).
[16] RABINER L R. A tutorial on hidden Markov models and selected applications in speech recognition[J]. Proceedings of the IEEE, 1989, 77(2):257-286.
[17] BREIMAN L. Random forests[J]. Machine learning, 2001, 45(1):5-32.
[18] 高金勇,徐朝军,冯奕竸.基于迭代的TFIDF在短文本分类中的应用[J].情报理论与实践,2011,34(6):120-122.
[19] 范云杰,刘怀亮.基于维基百科的中文短文本分类研究[J].现代图书情报技术,2012(3):47-52.
[20] MINAEE S, KALCHBRENNER N, CAMBRIA E, et al. Deep learning based text classification:a comprehensive review[J]. arXiv preprint arXiv:2004.03705, 2020.
[21] YIN W, KANN K, YU M, et al. Comparative study of cnn and rnn for natural language processing[J]. arXiv preprint arXiv:1702.01923, 2017.
[22] HOCHREITER S, SCHMIDHUBER J. Long short-term memory[J]. Neural computation, 1997, 9(8):1735-1780.
[23] OLAH C. Understanding lstm networks[EB/OL] [2015-8-27]. https://colah.github.io/posts/2015-08-Understanding-LSTMs/.
[24] CHO K, VAN MERRIÄNBOER B, GULCEHRE C, et al. Learning phrase representations using RNN encoder-decoder for statistical machine translation[J]. arXiv preprint arXiv:1406.1078, 2014.
[25] VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[C]//Advances in neural information processing systems. 2017:5998-6008.
[26] 邓三鸿,傅余洋子,王昊.基于LSTM模型的中文图书多标签分类研究[J].数据分析与知识发现,2017,1(7):52-60.
[27] 吕璐成,韩涛,周健,等.基于深度学习的中文专利自动分类方法研究[J].图书情报工作,2020,64(10):75-85.
[28] LEE J, DERNONCOURT F. Sequential short-text classification with recurrent and convolutional neural networks[J]. arXiv preprint arXiv:1603.03827, 2016.
[29] 秦成磊,章成志.基于层次注意力网络模型的学术文本结构功能识别[J].数据分析与知识发现,2020,4(11):26-42.
[30] 陶志勇,李小兵,刘影,等.基于双向长短时记忆网络的改进注意力短文本分类方法[J].数据分析与知识发现,2019,3(12):21-29.
[31] 余本功,朱梦迪.基于层级注意力多通道卷积双向GRU的问题分类研究[J].数据分析与知识发现,2020,4(8):50-62.
[32] DEVLIN J, CHANG M W, LEE K, et al. Bert:Pre-training of deep bidirectional transformers for language understanding[J]. arXiv preprint arXiv:1810.04805, 2018.
[33] SUN C, QIU X, XU Y, et al. How to fine-tune bert for text classification?[C]//China national conference on Chinese computational linguistics. Cham:Springer, 2019:194-206.
[34] LEE J S, HSIANG J. Patentbert:Patent classification with fine-tuning a pre-trained bert model[J]. arXiv preprint arXiv:1906.02124, 2019.
[35] LU X, NI B. BERT-CNN:A hierarchical patent classifier based on a pre-trained language model[J]. arXiv preprint arXiv:1911.06241, 2019.
[36] 刘欢,张智雄,王宇飞.BERT模型的主要优化改进方法研究综述[J/OL].数据分析与知识发现:1-17[2021-01-05]. https://doi.org/10.11925/infotech.2096-3467.2020.0965.
[37] LIU W, ZHOU P, ZHAO Z, et al. K-BERT:Enabling Language Representation with Knowledge Graph[J]. arXiv preprint arXiv:1909.07606, 2019.
[38] YU S, SU J, LUO D. Improving BERT-based text classification with auxiliary sentence and domain knowledge[J]. IEEE access, 2019, 7:176600-176612.
[39] ORKPHOL K, YANG W. Word sense disambiguation using cosine similarity collaborates with Word2vecand WordNet[J]. Future Internet, 2019, 11(5):114.
[40] MIKOLOV T, CHEN K, CORRADO G, et al. Efficient estimation of word representations in vector space[J]. arXiv preprint arXiv:1301.3781, 2013.
[41] 中国大百科全书总编辑委员会.中国大百科全书图书馆学·情报学·档案学[M].北京:中国大百科全书出版社, 2002.
[42] 丘东江. 新编图书馆学情报学辞典[M].北京:科学技术文献出版社, 2006.
[43] 白如江,张庆芝,孙一钢.科技文献知识基因表达及遗传与变异研究[J].图书情报工作,2020,64(4):78-87.
[44] 图书馆·情报与文献学名词审定委员会. 图书馆·情报与文献学名词2019[M].北京:科学出版社,2019.
[45] ASHKAN J, HAMED E, MIHAN H, et al. Improvement in automatic classification of Persian documents by means of support vector machine and representative vector[C]//International conference on innovative computing technology. Berlin:Springer, 2011:282-292.
[46] 杨敏,谷俊.基于SVM的中文书目自动分类及应用研究[J].图书情报工作,2012,56(9):114-119.
[47] 王东波,何琳,黄水清.基于支持向量机的先秦诸子典籍自动分类研究[J].图书情报工作,2017,61(12):71-76.
[48] WANG J H, LIU T W, LUO X, et al. An LSTM approach to short text sentiment classification with word embeddings[C]//Proceedings of the 30th conference on computational linguistics and speech processing (ROCLING 2018). Hsinchu:ACLCLP, 2018:214-223.

融合语义联想和BERT的图情领域SAO短文本分类研究

Research on SAO Short Text Classification in LIS Based on Semantic Association and BERT

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

编辑推荐

Metrics

[1]	吕璐成, 赵亚娟, 王学昭, 韩涛, 赵萍, 张迪. 基于表示学习的技术融合差异度测度方法及其效果研究[J]. 图书情报工作, 2022, 66(4): 118-128.
[2]	韩普, 顾亮. 基于混合深度学习的中文医学实体抽取研究[J]. 图书情报工作, 2022, 66(14): 119-127.
[3]	宗威, 林松涛, 刘继昶. 数据集成场景下考虑语义信息的高效实体识别方法[J]. 图书情报工作, 2022, 66(14): 128-136.
[4]	周潇, 许银彪, 史益. 基于深度学习与语义挖掘的技术创新组合识别与追踪[J]. 图书情报工作, 2022, 66(10): 33-44.
[5]	张国标, 李鹏程, 陆伟, 程齐凯. 多特征融合的关键词语义功能识别研究[J]. 图书情报工作, 2021, 65(9): 89-96.
[6]	梁媛, 王东波, 黄水清. 古籍同事异文的自动发掘研究[J]. 图书情报工作, 2021, 65(9): 97-104.
[7]	陆泉, 郝志同, 陈静, 陈仕, 朱安琪. 利用迁移学习精准识别领域信息之探讨[J]. 图书情报工作, 2021, 65(5): 110-117.
[8]	张敏, 刘欢, 丁良萍, 范青. 基于深度学习的网络科技信息情报价值计算方法研究[J]. 图书情报工作, 2021, 65(23): 70-78.
[9]	王秀红, 高敏. 基于BERT-LDA的关键技术识别方法及其实证研究——以农业机器人为例[J]. 图书情报工作, 2021, 65(22): 114-125.
[10]	任海英, 李真. 基于输入输出型SAO网络的核心技术链识别方法研究——以量子计算领域为例[J]. 图书情报工作, 2021, 65(19): 117-129.
[11]	杨波, 邵婉婷. 基于LDA-BERT融合模型的弱信号识别研究【涉嫌严重学术不端撤稿】[J]. 图书情报工作, 2021, 65(16): 98-107.
[12]	施国良, 陈宇奇. 文本增强与预训练语言模型在网络问政留言分类中的集成对比研究[J]. 图书情报工作, 2021, 65(13): 96-107.
[13]	罗卓然, 蔡乐, 钱佳佳, 陆伟. 学术论文创新贡献句识别研究[J]. 图书情报工作, 2021, 65(12): 93-100.
[14]	雷鸣, 夏梦鸽, 汪雪锋, 刘佳. 基于链路预测的协同药物组合推荐研究:面向疾病并发症诊疗[J]. 图书情报工作, 2021, 65(12): 122-129.
[15]	李晓曼, 张学福, 宋红燕, 孙巍. 专利文献技术要素识别方法研究——以纳米肥料领域为例[J]. 图书情报工作, 2020, 64(6): 59-68.