[Purpose/Significance] To reveal the core innovation points of literature, automatically generate multiple literature reviews, help researchers quickly grasp the core content of literature, and improve the efficiency of scientific research. [Method/Process] This paper proposed an automatic review research method for BERT scientific and technological literature based on knowledge gene attention enhancement, which was divided into three steps. Firstly, a core literature recommendation index was proposed to select literature review candidated by comprehensively considering topic similarity, publication time and citation times. Then the knowledge genes representing the core viewpoints of scientific and technological literature were extracted from literature review candidates. Finally, an automatic BERT scientific literature review model based on knowledge gene attention enhancement was proposed. Knowledge genes were integrated into the attention mechanism to judge the significance of sentences and sort and extract them to obtain more semantic information. [Result/Conclusion] After several sets of experiments, compared with BERT alone, Rouge-1 of the proposed model is improved by 14.28%, respectively. Rouge-2 is increased by 12.13%; Rouge-l is increased by 17.69% respectively. In the evaluation of Rouge-1 and Rouge-2, the automatic review model of BERT scientific and technological literature based on knowledge gene enhancement is better than TextRank model. Automatic review of BERT scientific and technological literature based on enhanced knowledge gene attention can dig into the text content, explore the core content of literature, and generate concise literature review.
[1] 中国科技论文统计与分析课题组.2020年中国科技论文统计与分析简报[J].中国科技期刊研究,2022,33(1):103-112.
[2] LUHN H P. The automatic creation of literature abstracts[J]. IBM journal of research and development, 1958, 2(2): 159-165.
[3] EDMUNDSON H P, WYLLYS R E. Automatic abstracting and indexing -survey and recommendations[J]. Communications of the ACM, 1961, 4(5): 226-234.
[4] SALTON G, YU C T. On the construction of effective vocabularies for information retrieval[J]. ACM sigplan notices, 1975, 10(1):48-60.
[5] KUPIEC J, PEDERSEN J O, CHEN F. A trainable document summarizer[C]// Proceedings of the 18th annual international ACM SIGIR conference on research and development in information retrieval. Seattle: ACM, 1995: 68-73.
[6] SUTSKEVER I, VINYALS O, LE Q V. Sequence to sequence learning with neural networks[C]// Proceedings of the 27th international conference on neural information processing systems. Cambridge: MIT Press, 2014: 3104-3112.
[7] BAHDANAU D, CHO K, BENGIO Y. Neural machine translation by jointly learning to align and translate [EB/OL]. [ 2022-10-11]. https://arxiv.org/pdf/1409.0473.pdf.
[8] VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[C]// Proceedings of the 31st international conference on neural information processing systems. Red Hook: Curran Associates, 2017: 6000-6010.
[9] EDMUNDSON H P. New methods in automatic extracting[J]. Journal of the ACM, 1969, 16(2): 264-285.
[10] NALLAPATI R, ZHAI F, ZHOU B W. Summarunner: a recurrent neural network based sequence model for extractive summarization of documents[C]// Proceedings of the thirty-first AAAI conference on artificial intelligence. Palo Alto: AAAI Press, 2017: 3075-3081.
[11] CONROY J M, O’LEARY D P. Text summarization via hidden markov models[C]//Proceedings of the 24th annual international ACM SIGIR conference on research and development in information retrieval. New York: ACM, 2001: 406-407.
[12] OSBORNE T J, NIELSEN M A. Entanglement in a simple quantum phase transition[J]. Physical review A, 2002, 66(3): 032110.
[13] LIU L, LU Y, YANG M, et al. Generative adversarial network for abstractive text summarization[C]//Proceedings of the 32nd AAAI conference on artificial intelligence. Menlo Park: AAAI, 2018: 8109-8110.
[14] SLAMET C, ATMADJA A R, MAYLAWATI D S, et al. Automated text summarization for indonesian article using vector space model [C]//IOP conference series materials science and engineering. Bandung: IOP publishing ltd, 2018: 32664-32671.
[15] CHO K, VAN MERRIËNBOER B, GULCEHRE C, et al. Learning phrase representations using RNN encoder-decoder for statistical machine translation[C]// Proceedings of the 2014 conference on empirical methods in natural language processing. Stroudsburg: Association for computational linguistics, 2014: 1724-1734.
[16] SIDDIQUI T, SHAMSI J A. Generating abstractive summaries using sequence to sequence attention[C]//Proceedings of the 2018 international conference on frontiers of information technology (FIT). Islamabad: IEEE Computer Society, 2018: 212-217.
[17] CELIKYILMAZ A, BOSSELUT A, HE X D, et al. Deep communicating agents for abstractive summarization[C]//Proceedings of the conference of the North American Chapter of the Association for Computational Linguistics: human language technologies. Stroudsburg: Association for Computational Linguistics, 2018: 1662-1675.
[18] 江跃华,丁磊,李娇娥,等.融合词汇特征的生成式摘要模型[J].河北科技大学学报,2019,40(2):152-158.
[19] 李维勇,柳斌,张伟,等.一种基于深度学习的中文生成式自动摘要方法[J].广西师范大学学报(自然科学版),2020,38(2):51-63.
[20] 吴世鑫,黄德根,李玖一.基于语义对齐的生成式文本摘要研究[J].北京大学学报(自然科学版),2021,57(1):1-6.
[21] 邱俊.基于强化学习的混合式文本摘要模型[J].信息技术与信息化,2019(1):67-70.
[22] 吕瑞,王涛,曾碧卿,等.TSPT:基于预训练的三阶段复合式文本摘要模型[J].计算机应用研究,2020,37(10):2917-2921.
[23] TEUFEL S, MOENS M. Summarizing scientific articles: experiments with relevance and rhetorical status[J]. Computational linguistics, 2002, 28(4): 409-445.
[24] 任潇雨. 基于引文的英文文档文摘自动生成方法研究[D].西安:西安电子科技大学,2014.
[25] NANBA H, OKUMURA M. Towards multi-paper summarization using reference information[C]// Proceedings of the sixteenth International Joint Conference on Artificial Intelligence. San Francisco: Morgan Kaufmann,1999: 926-913.
[26] HOANG C D V, KAN M Y. Towards automated related work summarization[C]// Coling 2010 - 23rd international conference on computational linguistics: posters. Stroudsburg: Association for Computational Linguistics, 2010: 427-435.
[27] AGARWAL N, REDDY R S, KIRAN G V R, et al. Towards multi-document summarization of scientific articles: making interesting comparisons with SciSumm[C]//Proceedings of the workshop on automatic summarization for different genres, media, and languages. Portland: Association for Computational Linguistics, 2011: 8-15.
[28] HU Y, WAN X. Automatic generation of related work sections in scientific papers: an optimization approach[C]//Proceedings of the 2014 conference on empirical methods in natural language processing. Doha: Association for Computational Linguistics, 2014: 1624-1633.
[29] 张占江.基于短语主题模型和多文档自动摘要技术的文献综述内容推荐[D].杭州: 浙江大学,2016.
[30] 唐晓波,翟夏普.基于混合机器学习模型的多文档自动摘要[J].情报理论与实践,2019,42(2):145-150.
[31] FERRARA F, PUDOTA N, TASSO C. A keyphrase-based paper recommender system[C]//Italian research conference on digital libraries. Heidelberg: Springer, 2011: 14-25.
[32] 刘旭晖.融合主题多样性与影响力的科技文献推荐算法研究[J].情报理论与实践,2017,40(12):134-138.
[33] SAYYADI H, GETOOR L. Futurerank: ranking scientific articles by predicting their future pagerank[C]//Proceedings of the 2009 SIAM international conference on data mining. USA: Society for Industrial and Applied Mathematics, 2009: 533-544.
[34] BERBERICH K, VAZIRGIANNIS M, WEIKUM G. Time-aware authority ranking[J]. Internet mathematics, 2005, 2(3): 301-332.
[35] 白如江,张庆芝,孙一钢.科技文献知识基因表达及遗传与变异研究[J].图书情报工作,2020,64(4):78-87.
[36] 刘明月,白如江,董坤,等.基于知识基因表达的智慧文献服务模式研究[J].情报理论与实践,2021,44(1):147-153.
[37] 白如江,孙一钢,张庆芝.基于知识基因表达的科技创新路径构建研究[J].情报理论与实践,2020,43(4):137-144,176.
[38] 刘植惠. 知识基因理论的由来, 基本内容及发展[J]. 情报理论与实践, 1998, 21(2): 71-76.
[39] ERKAN G, RADEV D R. Lexrank: graph-based lexical centrality as salience in text summarization[J]. Journal of artificial intelligence research, 2004, 22(1): 457-479.
[40] MIHALCEA R, TARAU P. Textrank: Bringing order into text[C]//Proceedings of the 2004 conference on empirical methods in natural language processing. Barcelona: Association for Computational Linguistics, 2004: 404-411.
[41] LIN C Y. Rouge: a package for automatic evaluation of summaries[C]// Proceedings of workshop on text summarization of ACL. Spain: ACL, 2004: 74-81.
