[1] 朱晓华, 张燕, 朱媛媛.健康中国战略: 从理论构想到实践推动[J]. 经济地理, 2023, 43(12): 1-12. (ZHU X H, ZHANG Y, ZHU Y Y. Healthy China strategy: from theoretical conception to practical promotion[J]. Economic geography, 2023, 43(12): 1-12.)
[2] 杨善林, 丁帅, 顾东晓, 等. 医疗健康大数据驱动的知识发现与知识服务方法[J]. 管理世界, 2022, 38(1): 219-229. (YANG S L, DING S, GU D X, et al. Healthcare big data driven knowledge discovery and knowledge service approach[J]. Journal of management world, 2022, 38(1): 219-229.)
[3] 单涛, 许鑫, 王园梦, 等. 面向中文病历的实体关系抽取模型研究[J/OL]. 现代情报, 1-15[2024-08-28]. http://kns.cnki.net/kcms/detail/22.1182.G3.20240523.1258.002.html. (SHAN T, XU X, WANG M Y, et al. Research on entity relationship extraction model for Chinese medical record[J]. Journal of modern information, 1-15[2024-08-28]. http://kns.cnki.net/kcms/detail/22.1182.G3.20240523.1258.002.html.)
[4] 张传洋, 郭宇, 庞宇飞, 等. 数智化医疗信息利用与服务模式框架构建[J]. 图书情报工作, 2023, 67(13): 49-58. (ZHANG C Y, GUO Y, PANG Y F, Construction of medical information utilization and service framework for digital and intellectual transformation[J]. Library and information service, 2023, 67(13): 49-58.)
[5] 王萌, 王昊奋, 李博涵, 等. 新一代知识图谱关键技术综述[J]. 计算机研究与发展, 2022, 59(9): 1947-1965. (WANG M, WANG H F, LI B H, et al. Survey on key technologies of new generation knowledge graph[J]. Journal of computer research and development, 2022, 59(9): 1947-1965.)
[6] 刘道文, 阮彤, 张晨童, 等. 基于多源知识图谱融合的智能导诊算法[J]. 中文信息学报, 2021, 35(1): 125-134. (LIU D W, RUAN T, ZHANG C T, et al. Clinical departments recommendation by fusing knowledge graphs from electronic healthcare records and medical websites[J]. Journal of Chinese information processing, 2021, 35(1): 125-134.)
[7] 李贺, 祝琳琳, 刘嘉宇, 等. 基于本体的简帛医药知识组织研究[J]. 图书情报工作, 2022, 66(22): 16-27. (LI H, ZHU L L, LIU J Y, et al. Research on the organization of bamboo and silk medical knowledge based on ontology[J]. Library and information service, 2022, 66(22): 16-27.)
[8] 赵哲焕, 杨志豪, 孙聪, 等. 生物医学文献中的蛋白质关系抽取研究[J]. 中文信息学报, 2018, 32(7): 82-90. (ZHAO Z H, YANG Z H, SUN C, et al. Protein-Protein interaction extraction from biomedical literature[J]. Journal of Chinese information processing, 2018, 32(7): 82-90.)
[9] 韩普, 叶东宇, 陈文祺, 等. 面向多模态医疗健康数据的知识组织模式研究[J]. 现代情报, 2023, 43(10): 27-34. (HAN P, YE D Y, CHEN W Q, et al. Research on the knowledge organization model of multimodal medical and health data[J]. Journal of modern information, 2023, 43(10): 27-34.)
[10] BALTRUŠAITIS T, AHUJA C, MORENCY L P. Multimodal machine learning: a survey and taxonomy[J]. IEEE trans on pattern analysis and machine intelligence, 2018, 41(2): 423-443.
[11] ZHU X, LI Z, WANG X, et al. Multi-Modal knowledge graph construction and application: a survey[J]. IEEE transactions on knowledge and data engineering, 202236(2): 715-735.
[12] MORENCY L P, LIANG P P, ZADEH A. Tutorial on multimodal machine learning[C]//Proceedings of the 2022 conference of the North American Chapter of the Association for Computational Linguistics: human language technologies: tutorial abstracts. Stroudsburg: ACL, 2022: 33-38.
[13] ZHANG C, YANG Z, HE X, et al. Multimodal intelligence: Representation learning, information fusion, and applications[J]. IEEE journal of selected topics in signal processing, 2020, 14(3): 478-493.
[14] LIANG P P, ZADEH A, MORENCY L P. Foundations and recent trends in multimodal machine learning: principles, challenges, and open questions[J]. arXiv preprint, 2022, arXiv:2209.03430.
[15] 吴友政, 李浩然, 姚霆, 等. 多模态信息处理前沿综述: 应用、融合和预训练[J]. 中文信息学报, 2022, 36(5): 1-20. (WU Y Z, LI H R, YAO T, et al. A survey of multimodal information processing frontiers: application, fusion and pre-training[J]. Journal of Chinese information processing, 2022, 36(5): 1-20.)
[16] GUO W, ZHANG Y, YANG J, et al. Re-attention for visual question answering[J]. IEEE transactions on image processing, 2021, 30: 6730-6743.
[17] ZHANG J, YIN Z, CHEN P, et al. Emotion recognition using multi-modal data and machine learning techniques: a tutorial and review[J]. Information fusion, 2020, 59: 103-126.
[18] 范涛, 王昊, 陈玥彤. 基于深度迁移学习的地方志多模态实体识别研究[J]. 情报学报, 2022, 41(4): 412-423. (FAN T, WANG H, CHEN Y T. Research on multimodal named entity recognition of local history based on deep transfer learning[J]. Journal of the China Society for Scientific and Technical Information, 2022, 41(4): 412-423.)
[19] JANGRA A, MUKHERJEE S, JATOWT A, et al. A survey on multi-modal summarization[J]. ACM computing surveys, 2023, 55(13): 1-36.
[20] MOON S, NEVES L, CARVALHO V. Multimodal named entity recognition for short social media posts[C]//Proceedings of the 2018 conference of the North American Chapter of the Association for Computational Linguistics: human language technologies. Stroudsburg: ACL, 2018: 852-860.
[21] ZHENG C, WU Z, FENG J, et al. Mnre: A challenge multimodal dataset for neural relation extraction with visual evidence in social media posts[C]//2021 IEEE international conference on multimedia and expo. Piscataway: IEEE, 2021: 1-6.
[22] XU B, HUANG S, SHA C, et al. MAF: a general matching and alignment framework for multimodal named entity recognition[C]//Proceedings of the fifteenth ACM international conference on Web search and data mining. New York: ACM, 2022: 1215-1223.
[23] 张天明, 张杉, 刘曦, 等. 融合多模态数据的小样本命名实体识别方法[J]. 软件学报, 2024, 35(3): 1107-1124. (ZHANG T M, ZHANG S, LIU X, et al. Multimodal data fusion for few-shot named entity recognition method[J]. Journal of software, 2024, 35(3): 1107-1124.)
[24] ZHENG C, FENG J, FU Z, et al. Multimodal relation extraction with efficient graph alignment[C]//Proceedings of the 29th ACM international conference on multimedia. New York: ACM, 2021: 5298-5306.
[25] ZHAO Q, GAO T, GUO N. TSVFN: Two-Stage visual fusion network for multimodal relation extraction[J]. Information processing & management, 2023, 60(3): 103264.
[26] ZHOU H Y, YU Y, WANG C, et al. A transformer-based representation-learning model with unified processing of multimodal input for clinical diagnostics[J]. Nature biomedical engineering, 2023, 7(6): 743-755.
[27] ZHENG W, YAN L, GOU C, et al. Pay attention to doctor-patient dialogues: multi-modal knowledge graph attention image-text embedding for COVID-19 diagnosis[J]. Information fusion, 2021, 75: 168-185.
[28] XIA F, LI B, WENG Y, et al. LingYi: Medical conversational question answering system based on multi-modal knowledge graphs[J]. arXiv preprint, 2022, arXiv:2204.09220.
[29] LIU Y, LI H, GARCIA-DURAN A, et al. MMKG: multi-modal knowledge graphs[C]//The semantic Web: 16th international conference, ESWC 2019. Berlin: Springer, 2019: 459-474.
[30] OÑORO-RUBIO D, NIEPERT M, GARCÍA-DURÁN A, et al. Answering visual-relational queries in web-extracted knowledge graphs[J]. arXiv preprint, 2017, arXiv:1709.02314.
[31] WANG M, WANG H, QI G, et al. Richpedia: a large-scale, comprehensive multi-modal knowledge graph[J]. Big data research, 2020, 22: 100159.
[32] ALBERTS H, HUANG N, DESHPANDE Y R, et al. VisualSem: A high-quality knowledge graph for vision & language[C]//1st Workshop on Multilingual Representation Learning. Stroudsburg: Association for Computational Linguistics, 2021: 138-152.
[33] CHANDAK P, HUANG K, ZITNIK M. Building a knowledge graph to enable precision medicine[J]. Scientific data, 2023, 10(1): 67-82.
[34] 张云秋, 汪洋, 李博诚. 基于RoBERTa-wwm动态融合模型的中文电子病历命名实体识别[J]. 数据分析与知识发现, 2022, 6(Z1): 242-250. (ZHANG Y Q, WANG Y, LI B C. Identifying named entities of Chinese electronic medical records based on RoBERTa-wwm dynamic fusion model[J]. Data analysis and knowledge discovery, 2022, 6(Z1): 242-250.)
[35] HIROKI N, TAKAHIRO K, JUNYA K, et al. Doccano: Text annotation tool for human[EB/OL]. [2024-08-27]. https://github.com/doccano/doccano.
[36] 张仰森, 刘帅康, 刘洋, 等. 基于深度学习的实体关系联合抽取研究综述[J]. 电子学报, 2023, 51(4): 1093-1116. (ZHANG Y S, LIU S K, LIU Y, et al. Joint extraction of entities and relations based on deep learning: a survey[J]. Acta electronica sinica, 2023, 51(4): 1093-1116.)
[37] WANG F, ZHOU Y, WANG S, et al. Multi-granularity cross-modal alignment for generalized medical visual representation learning[J]. Advances in neural information processing systems, 2022, 35: 33536-33549.
[38] FAN W, XIAO F. A complex Jensen–Shannon divergence in complex evidence theory with its application in multi-source information fusion[J]. Engineering applications of artificial intelligence, 2022, 116: 105362.
[39] ATTERER M, SCHÜTZE H. Prepositional phrase attachment without oracles[J]. Computational linguistics, 2007, 33(4): 469-476.
[40] LI L H, YATSKAR M, YIN D, et al. Visualbert: A simple and performant baseline for vision and language[J]. arXiv preprint, 2019, arXiv:1908.03557.
[41] YU J, JIANG J, YANG L, et al. Improving multimodal named entity recognition via entity span detection with unified multimodal transformer[C]//Proceedings of the 58th annual meeting of the Association for Computational Linguistics. Stroudsburg: ACL, 2020: 3342-3352.
[42] ZHANG D, WEI S, LI S, et al. Multi-modal graph fusion for named entity recognition with targeted visual guidance[C]//Proceedings of the AAAI conference on artificial intelligence. Menlo Park: AAAI, 2021, 35(16): 14347-14355.
[43] ZHENG C, WU Z, WANG T, et al. Object-aware multimodal named entity recognition in social media posts with adversarial learning[J]. IEEE transactions on multimedia, 2020, 23: 2520-2532.
[44] CHEN X, ZHANG N, LI L, et al. Good visual guidance make a better extractor: hierarchical visual prefix for multimodal entity and relation extraction[C]//Findings of the Association for Computational Linguistics: NAACL 2022. Stroudsburg: ACL, 2022: 1607-1618.
[45] HUANG Y, LIN Z. I2SRM: Intra- and inter-sample relationship modeling for multimodal information extraction[J]. arXiv preprint, 2023, arXiv:2310.06326.
[46] 刘任烨, 于凯. 面向情报感知的领域知识图谱构建研究与应用[J]. 情报理论与实践, 2024, 47(4): 27-34. (LIU R Y, YU K. Research and application of domain knowledge graph construction for information awareness[J]. Information studies: theory & application, 2024, 47(4): 27-34.)
[47] 吴梦成, 林立涛, 齐月, 等. 数字人文视域下先秦典籍植物知识挖掘与组织研究[J]. 图书情报工作, 2023, 67(12): 103-113. (WU M C, LIN L T, QI Y, et al. Plant knowledge mining and organization construction in pre-Qin classics from the perspective of digital humanities[J]. Library and information service, 2023, 67(12): 103-113.)
[48] HAFNER M, KATSANTONI M, KÖSTER T, et al. CLIP and complementary methods[J]. Nature reviews methods primers, 2021, 1(1): 1-23.
[49] 席运江, 李曼, 邓雨珊, 等. 中文在线医疗社区问答内容知识图谱构建研究[J]. 图书情报工作, 2024, 68(4): 124-136. (XI Y J, LI M, DENG Y S, et al. A knowledge graph construction for Q&A text in Chinese online medical community[J]. Library and information service, 2024, 68(4): 124-136.)
[50] 王燕红, 司徒凌云, 杨海平, 等.基于知识图谱的书证目录知识发现研究——以南海书证目录为例[J]. 情报杂志, 2022, 41(3): 173-180. (WANG Y H, SI TU L Y, YANG H P, et al. Research on knowledge discovery of documentary evidence catalog based on knowledge map: taking the south China sea document evidence catalog as an example[J]. Journal of intelligence, 2022, 41(3): 173-180.)
[51] 张云中, 李茜. 沪上名人故居知识图谱构建与应用研究[J]. 情报科学, 2023, 41(10): 1-11. (ZHANG Y Z, LI X. The construction and application of knowledge map of the former residences of celebrities in Shanghai[J]. Information science, 2023, 41(10): 1-11.)
[52] 何巍. 社交网络舆情多模态知识图谱构建框架研究[J]. 情报杂志, 2024, 43(1): 160-166. (HE W. Research on the construction framework of multi-modal knowledge graph for social network public opinion[J]. Journal of intelligence, 2024, 43(1): 160-166.)
