Library and Information Service

Library and Information Service >

2025 , Vol. 69 >Issue 1: 80 - 91

DOI: https://doi.org/10.13266/j.issn.0252-3116.2025.01.008

RESEARCH PAPERS

Research on Visual Interpretation of BERT for Intelligence Analysis Based on Gradient Saliency

  • Zhang Tao ,
  • Ma Haiqun ,
  • Jiang Lei
Expand
  • 1 School of Information Management, Heilongjiang University, Harbin 150080;
    2 Research center of information resource management, Heilongjiang University, Harbin 150080
Zhang Tao, professor, PhD, master supervisor; Ma Haiqun, professor, PhD, doctoral supervisor, corresponding author, E-mail: mahaiqun@sina.com.cn; Jiang Lei, master candidate, engineer.

Received date: 2024-01-26

  Revised date: 2024-05-06

  Online published: 2025-01-15

Supported by

This work is supported by the project of National Social Science Fund of China titled “Research on the Risk Governance Path of Intelligence Analysis Algorithms in Data Intelligence Environment” (Grant No. 22BTQ064).

Fold

Abstract

[Purpose/Significance] As pre-trained models continue to demonstrate astonishing capabilities, more and more researchers are introducing BERT into the field of intelligence analysis, leading to a trend of evolution from a single BERT model to a fused BERT model. However, the problem of inexplicable prediction results of pre-trained language models poses some risks to the algorithms generalization in the field of intelligence analysis. [Method/Process] This article took the BERT algorithm, which was widely used in intelligence analysis, as an example. Using significance theory, it calculated the Token Embedding significance of the hidden layer state values inside BERT and explored the key factors that affected the final classification results. Finally, taking the team data security policy classification intelligence analysis project as an example, it visualized and deeply analyzed the BERT’s interpretability in it. [Result/Conclusion] The visualized interpretation model, based on gradient saliency, provides clear insights into the running status of each BERT layer. By re-labeling misclassified samples, the accuracy of the validation set is improved from 96.74% to 97.78%. It also indicates that this method can effectively guide the dataset annotation in policy text classification tasks and provide reference for the wider application of this model in the field of intelligence analysis.

Key words： interpretability; BERT; deep learning; intelligence analysis algorithms; gradient significance

Cite this article

Zhang Tao , Ma Haiqun , Jiang Lei . Research on Visual Interpretation of BERT for Intelligence Analysis Based on Gradient Saliency[J]. Library and Information Service, 2025 , 69(1) : 80 -91 . DOI: 10.13266/j.issn.0252-3116.2025.01.008

References

[1] 刘如, 张惠娜, 周京艳. 科技决策智能情报[M]. 北京: 科学技术文献出版社, 2022. (LIU R, ZHANG H N, ZHOU J Y. S&T decision-making and smart intelligence[M]. Beijing: Scientific and Technical Documentation Press, 2022.)
[2] 张涛, 马海群. 智能情报分析中算法风险及其规制研究[J]. 图书情报工作, 2021, 65(12): 47-56. (ZHANG T, MA H Q. Research on algorithm risk and regulation in intelligent intelligence analysis[J]. Library and information service, 2021, 65(12): 47-56.)
[3] 靳庆文. 情报分析中的可解释性技术及其评价方法研究[J]. 情报资料工作, 2023, 44(4): 24-34. (JIN Q W. Research on interpretability technology and evaluation method in information analysis[J]. Information and documentation services, 2023, 44(4): 24-34.)
[4] 陆伟, 李鹏程, 张国标, 等. 学术文本词汇功能识别—基于BERT向量化表示的关键词自动分类研究[J]. 情报学报, 2020, 39(12): 1320-1329. (LU W, LI P C, ZHANG G B, et al. Recognition of lexical functions in academic texts: automatic classification of keywords based on BERT vectorization[J]. Journal of the China Society for Scientific and Technical Information, 2020, 39(12): 1320-1329.)
[5] 翟羽佳, 田静文, 赵玥. 基于BERT-BiLSTM-CRF模型的算法术语抽取与创新演化路径构建研究[J]. 情报科学, 2022, 40(4): 71-78. (ZHAI Y J, TIAN J W, ZHAO Y, Research on algorithm term extraction and innovation evolution path construction based on BERT-BiLSTM-CRF model[J]. Information science, 2022, 40(4): 71-78.)
[6] 马雨萌, 黄金霞, 王昉, 等. 融合BERT与多尺度CNN的科技政策内容多标签分类研究[J]. 情报杂志, 2022, 41(11): 157-163. (MA Y M, HUANG J X, WANG F, et al. Research on multi-label classification of S&T policy content combining BERT and multi-scale CNN[J]. Journal of intelligence, 2022, 41(11): 157-163.)
[7] 霍朝光, 霍帆帆, 王婉如, 等. 基于WordBERT和BiLSTM的政策工具自动分类方法研究[J]. 图书情报知识, 2023, 40(3): 129-138. (HUO C G, HUO F F, WANG W R, et al. Automatic classification method of policy tools based on WordBERT and BiLSTM[J]. Document, information & knowledge, 2023, 40(3): 129-138.)
[8] 沈思, 陈猛, 冯暑阳, 等. ChpoBERT:面向中文政策文本的预训练模型[J]. 情报学报, 2023, 42(12): 1487-1497. (SHEN S, CHEN M, FENG S Y, et al. ChpoBERT: a pre-trained model for Chinese policy texts[J]. Journal of the China Society for Scientific and Technical Information, 2023, 42(12): 1487-1497.)
[9] XIE Q, ZHANG X, DING Y, et al. Monolingual and multilingual topic analysis using LDA and BERT embeddings [J]. Journal of informetrics.2020, 14(3): 101055.
[10] MILES S, YAO L, MENG W. Comparing PSO-based clustering over contextual vector embeddings to modern topic modeling[J]. Information processing & management.2022, 59(3): 102921.
[11] LAMSIYAH S, MAHDAOUY A, ALAOUI S. Unsupervised extractive multi-document summarization method based on transfer learning from BERT multi-task fine-tuning[J]. Journal of information science.2023, 49(1): 164-182.
[12]CHENGZHI Z, YI X, WENKE H, et al. Automatic recognition and classification of future work sentences from academic articles in a specific domain[J]. Journal of informetrics.2023, 17(1): 101373.
[13] 梁峥, 王宏志, 戴加佳, 等.预训练语言模型实体匹配的可解释性[J]. 软件学报, 2023, 34(3): 1087-1108. (LIANG Z, WANG H Z, DAI J J, et al. Interpretability of entity matching based on pre-trained language model[J]. Journal of software, 2023, 34(3): 1087-1108.)
[14] 李伟卿, 池毛毛, 王伟军.面向用户长短期偏好调节的可解释个性化推荐方法研究[J]. 图书情报工作, 2021, 65(12): 101-111. (LI W Q, CHI M M, WANG W J. Explainable personalized recommendation method based on adjustment of users’ long- and short-term preferences[J]. Library and information service, 2021, 65(12): 101-111.)
[15] 易明, 姚玉佳, 胡敏.融合XGBoost与SHAP的政务新媒体公共价值共识可解释性模型——以“今日头条”十大市级政务号为例[J]. 图书情报工作, 2022, 66(16): 36-47. (YI M, YAO Y J, HU M. An Interpretable model for new government media public value consensus integrating XGBoost and SHAP: taking the top 10 municipal government accounts of the Jinri Toutiao as an example[J]. Library and information service, 2022, 66(16): 36-47.)
[16] PRUTHI D, GUPTA M, DHINGRA B, et al. Learning to deceive with attention-based explanations[J]. arXiv preprint arXiv:1909.07913, 2020.
[17] MEISTER C, LAZOV S, AUGENSTEIN I, et al.Is sparse attention more interpretable?[J]. arXiv preprint arXiv:2106.01087, 2021.
[18] HAO Y, DONG L, WEI F, et al. Self-attention attribution: Interpreting information interactions inside transformer[J]. arXiv preprint arXiv:2004.11207v1, 2021.
[19] EBAID A, THIRUMURUGANATHAN S, AREF W G, et al. Explainer: entity resolution explanations[C]//2019 IEEE 35th international conference on data engineering (ICDE). Macao: IEEE, 2019: 2000-2003.
[20] PEETERS R, BIZER C. Dual-objective fine-tuning of BERT for entity matching[C]//47th international conference on Very Large Data Bases. Copenhagen: ACM, 2021: 1913-1921.
[21] 肖纪文.面向局部可解释性机器学习的数据故事生成方法研究[J]. 图书情报工作, 2023, 67(2): 98-107. (XIAO J W. Research on the method of data story generation for local interpretable machine learning[J]. Library and information service, 2023, 67(2): 98-107.)
[22] WALLACE E, FENG S, KANDPAL N, et al. Universal adversarial triggers for attacking and analyzing NLP[J]. arXiv preprint arXiv:1908.07125, 2019.
[23] VALIPOUR M, LEE E S A, JAMACARO J R, et al.Unsupervised transfer learning via BERT neuron selection[J]. arXiv preprint arXiv:1912.05308, 2019.
[24] 高广尚.可解释推荐模型中的可解释性方法研究综述[J/OL]. 数据分析与知识发现:1-17[2024-01-25]. http://kns.cnki.net/kcms/detail/10.1478.G2.20240117.1116.026.html. (GAO G S. A survey of explainability methods in explainable recommendation models[J/OL]. Data analysis and knowledge discovery:1-17[2024-01-25]. http://kns.cnki.net/kcms/detail/10.1478.G2.20240117.1116.026.html.)
[25] ATANASOVA P, SIMONSEN J G, LIOMA C, et al. A diagnostic study of explainability techniques for text classification[J]. arXiv preprint arXiv:2009.13295, 2020.
[26] 张涛, 马海群.智能情报分析中数据与算法风险识别模型构建研究[J]. 情报学报, 2022, 41(8): 832-844. (ZHANG T, MA H Q. Research on the construction of data and algorithm risk identification model in intelligent intelligence analysis[J]. Journal of the China Society for Scientific and Technical Information, 2022, 41(8): 832-844.)
[27] BASTINGS J, FILIPPOVA K. The elephant in the interpretability room: why use attention as explanation when we have saliency methods?[J]. arXiv preprint arXiv:2010.05607, 2020.
[28] DEVLIN J, CHANG M W, LEE K, et al, BERT: pre-training of deep bidirectional transformers for language understanding[J]. arXiv preprint arXiv:1810.04805, 2019.
[29] 马海群, 张涛. 文献信息视阈下面向智慧服务的语料库构建研究[J]. 情报理论与实践, 2019, 42(6): 124-130. (ZHANG T, MA H Q. Research on the construction of smart service oriented corpus from the perspective of literature information[J]. Information studies: theory & application, 2019, 42(6): 124-130.)
[30] Clue benchmark [EB/OL]. [2024-07-11]. https://storage.googleapis.com/cluebenchmark/tasks/tnews_public.zip.
Options
Outlines

/

〈 〉

    Copyrights © LIS Press Co., Ltd. Reproduction is prohibited without permission.