Library and Information Service

Library and Information Service >

2021 , Vol. 65 >Issue 16: 118 - 129

DOI: https://doi.org/10.13266/j.issn.0252-3116.2021.16.013

KNOWLEDGE ORGANIZATION

Research on SAO Short Text Classification in LIS Based on Semantic Association and BERT

  • Zhang Yujie ,
  • Bai Rujiang ,
  • Liu Mingyue ,
  • Yu Chunliang
Expand
  • 1 Institute of Information Management, Shandong University of Technology, Zibo 255049;
    2 Yantai University Library, Yantai 264005

Received date: 2021-01-27

  Revised date: 2021-05-12

  Online published: 2021-08-20

Fold

Abstract

[Purpose/significance] Aiming at the shortage of semantic features and insufficient domain knowledge in the classification of SAO structure short texts, this paper proposes a SAO classification method combining semantic association and BERT in order to improve the classification effect.[Method/process] Taking the SAO short text in the library and information science field as the data source, firstly, a semantic association scheme including the three links of "Expansion-Reconstruction-NoiseReduction" was designed. The semantic information of SAO was extended through semantic expansion and SAO reconstruction, and the extended noise interference problem was solved by semantic noise reduction; then used the BERT model to train the SAO short text after semantic association; finally realized automatic classification in the classification part.[Result/conclusion] After comparing different association values, learning rates and classifiers, the experimental results show that when the association value is 10 and the learning rate is 4e-5, the SAO short text classification effect is optimal, and the average F1 value is 0.852 2, which is comparable to SVM and LSTM compared with pure BERT, the F1 value is increased by 0.103 1, 0.153 8 and 0.140 5 respectively.

Key words： SAO; short text classification; semantic association; BERT

Cite this article

Zhang Yujie , Bai Rujiang , Liu Mingyue , Yu Chunliang . Research on SAO Short Text Classification in LIS Based on Semantic Association and BERT[J]. Library and Information Service, 2021 , 65(16) : 118 -129 . DOI: 10.13266/j.issn.0252-3116.2021.16.013

References

[1] CASCINI G, FANTECHI A, SPINICCI E. Natural language processing of patents and technical documentation[C]//International workshop on document analysis systems. Berlin:Springer, 2004:508-520.
[2] CHOI S, PARK H, KANG D, et al. An sao-based text mining approach to building a technology tree for technology planning[J]. Expert systems with applications, 2012, 39(13):11443-11455.
[3] WANG X, WANG Z, HUANG Y, et al. Identifying r&d partners through subject-action-object semantic analysis in a problem & solution pattern[J]. Technology analysis & strategic management, 2017, 29(10):1167-1180.
[4] TSOURIKOV V M, BATCHILO L S, SOVPEL I V. Document semantic analysis/selection with knowledge creativity capability utilizing subject-action-object (sao) structures:U.S. Patent 6,167,370[P]. 2000-12-26.
[5] 付芸,汪雪锋,李佳,等.基于SAO结构的创新解决方案遴选研究——以空气净化技术为例[J].图书情报工作,2019,63(6):75-84.
[6] 许海云,王振蒙,胡正银,等.利用专利文本分析识别技术主题的关键技术研究综述[J].情报理论与实践,2016,39(11):131-137.
[7] 胡正银,刘春江,隗玲,等,文奕.面向TRIZ的领域专利技术挖掘系统设计与实践[J].图书情报工作,2017,61(1):117-124.
[8] 杨超,朱东华,汪雪锋,等.专利技术主题分析:基于SAO结构的LDA主题模型方法[J].图书情报工作,2017,61(3):86-96.
[9] CHANG P L, WU C C, Leu H J. Using patent analyses to monitor the technological trends in an emerging field of technology:a case of carbon nanotube field emission display[J]. Scientometrics, 2010, 82(1):5-19.
[10] GUO J, WANG X, LI Q, et al. Subject-action-object-based morphology analysis for determining the direction of technological change[J]. Technological forecasting and social change, 2016, 105:27-40.
[11] LI X, WANG J J, YANG Z. Identifying emerging technologies based on subject-action-object[J]. Journal of intelligence, 2016, 35(3):80-84.
[12] 王晓宇,苗红,王芳.技术知识的跨领域应用及潜在技术方案的识别[J].图书情报工作,2016,60(23):87-96.
[13] 胡正银,方曙,张娴,等.个性化语义TRIZ构建研究[J].图书情报工作,2015,59(7):123-131.
[14] MANEK A S, SHENOY P D, MOHAN M C, et al. Aspect term extraction for sentiment analysis in large movie reviews using gini index feature selection method and SVM classifier[J]. World Wide Web,2017, 20(2):135-154.
[15] BACHHETY S, DHINGRA S, JAIN R, et al. Improved multinomial naïve bayes approach for sentiment analysis on social media[J]. International journal of information systems & management science, 2018, 1(1).
[16] RABINER L R. A tutorial on hidden Markov models and selected applications in speech recognition[J]. Proceedings of the IEEE, 1989, 77(2):257-286.
[17] BREIMAN L. Random forests[J]. Machine learning, 2001, 45(1):5-32.
[18] 高金勇,徐朝军,冯奕竸.基于迭代的TFIDF在短文本分类中的应用[J].情报理论与实践,2011,34(6):120-122.
[19] 范云杰,刘怀亮.基于维基百科的中文短文本分类研究[J].现代图书情报技术,2012(3):47-52.
[20] MINAEE S, KALCHBRENNER N, CAMBRIA E, et al. Deep learning based text classification:a comprehensive review[J]. arXiv preprint arXiv:2004.03705, 2020.
[21] YIN W, KANN K, YU M, et al. Comparative study of cnn and rnn for natural language processing[J]. arXiv preprint arXiv:1702.01923, 2017.
[22] HOCHREITER S, SCHMIDHUBER J. Long short-term memory[J]. Neural computation, 1997, 9(8):1735-1780.
[23] OLAH C. Understanding lstm networks[EB/OL] [2015-8-27]. https://colah.github.io/posts/2015-08-Understanding-LSTMs/.
[24] CHO K, VAN MERRIÄNBOER B, GULCEHRE C, et al. Learning phrase representations using RNN encoder-decoder for statistical machine translation[J]. arXiv preprint arXiv:1406.1078, 2014.
[25] VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[C]//Advances in neural information processing systems. 2017:5998-6008.
[26] 邓三鸿,傅余洋子,王昊.基于LSTM模型的中文图书多标签分类研究[J].数据分析与知识发现,2017,1(7):52-60.
[27] 吕璐成,韩涛,周健,等.基于深度学习的中文专利自动分类方法研究[J].图书情报工作,2020,64(10):75-85.
[28] LEE J, DERNONCOURT F. Sequential short-text classification with recurrent and convolutional neural networks[J]. arXiv preprint arXiv:1603.03827, 2016.
[29] 秦成磊,章成志.基于层次注意力网络模型的学术文本结构功能识别[J].数据分析与知识发现,2020,4(11):26-42.
[30] 陶志勇,李小兵,刘影,等.基于双向长短时记忆网络的改进注意力短文本分类方法[J].数据分析与知识发现,2019,3(12):21-29.
[31] 余本功,朱梦迪.基于层级注意力多通道卷积双向GRU的问题分类研究[J].数据分析与知识发现,2020,4(8):50-62.
[32] DEVLIN J, CHANG M W, LEE K, et al. Bert:Pre-training of deep bidirectional transformers for language understanding[J]. arXiv preprint arXiv:1810.04805, 2018.
[33] SUN C, QIU X, XU Y, et al. How to fine-tune bert for text classification?[C]//China national conference on Chinese computational linguistics. Cham:Springer, 2019:194-206.
[34] LEE J S, HSIANG J. Patentbert:Patent classification with fine-tuning a pre-trained bert model[J]. arXiv preprint arXiv:1906.02124, 2019.
[35] LU X, NI B. BERT-CNN:A hierarchical patent classifier based on a pre-trained language model[J]. arXiv preprint arXiv:1911.06241, 2019.
[36] 刘欢,张智雄,王宇飞.BERT模型的主要优化改进方法研究综述[J/OL].数据分析与知识发现:1-17[2021-01-05]. https://doi.org/10.11925/infotech.2096-3467.2020.0965.
[37] LIU W, ZHOU P, ZHAO Z, et al. K-BERT:Enabling Language Representation with Knowledge Graph[J]. arXiv preprint arXiv:1909.07606, 2019.
[38] YU S, SU J, LUO D. Improving BERT-based text classification with auxiliary sentence and domain knowledge[J]. IEEE access, 2019, 7:176600-176612.
[39] ORKPHOL K, YANG W. Word sense disambiguation using cosine similarity collaborates with Word2vecand WordNet[J]. Future Internet, 2019, 11(5):114.
[40] MIKOLOV T, CHEN K, CORRADO G, et al. Efficient estimation of word representations in vector space[J]. arXiv preprint arXiv:1301.3781, 2013.
[41] 中国大百科全书总编辑委员会.中国大百科全书图书馆学·情报学·档案学[M].北京:中国大百科全书出版社, 2002.
[42] 丘东江. 新编图书馆学情报学辞典[M].北京:科学技术文献出版社, 2006.
[43] 白如江,张庆芝,孙一钢.科技文献知识基因表达及遗传与变异研究[J].图书情报工作,2020,64(4):78-87.
[44] 图书馆·情报与文献学名词审定委员会. 图书馆·情报与文献学名词2019[M].北京:科学出版社,2019.
[45] ASHKAN J, HAMED E, MIHAN H, et al. Improvement in automatic classification of Persian documents by means of support vector machine and representative vector[C]//International conference on innovative computing technology. Berlin:Springer, 2011:282-292.
[46] 杨敏,谷俊.基于SVM的中文书目自动分类及应用研究[J].图书情报工作,2012,56(9):114-119.
[47] 王东波,何琳,黄水清.基于支持向量机的先秦诸子典籍自动分类研究[J].图书情报工作,2017,61(12):71-76.
[48] WANG J H, LIU T W, LUO X, et al. An LSTM approach to short text sentiment classification with word embeddings[C]//Proceedings of the 30th conference on computational linguistics and speech processing (ROCLING 2018). Hsinchu:ACLCLP, 2018:214-223.
Options
Outlines

/

〈 〉

    Copyrights © LIS Press Co., Ltd. Reproduction is prohibited without permission.