图书情报工作

图书情报工作 >

2024 , Vol. 68 >Issue 24: 27 - 37

DOI: https://doi.org/10.13266/j.issn.0252-3116.2024.24.003

专题：多模态数据驱动的安全态势感知研究

多模态数据驱动的AI智能体模式设计

  • 韩雪雯 ,
  • 车尚锟 ,
  • 杨梦晴 ,
  • 王能
展开
  • 1 清华大学经济管理学院, 北京 100084;
    2 南京师范大学新闻与传播学院, 南京 210023;
    3 加州大学洛杉矶分校, 洛杉矶 90095
韩雪雯，博士研究生；车尚锟，博士研究生；杨梦晴，讲师，博士，通信作者，E-mail:mqyang@nnu.edu.cn；王能，硕士研究生。

收稿日期: 2024-06-21

  修回日期: 2024-09-28

  网络出版日期: 2024-12-23

基金资助

本文系国家社会科学基金一般项目“多模态网络数据安全态势感知及风险协同治理机制研究”（项目编号：23BTQ076）研究成果之一。

收起

Design of AI Agent Models Driven by Multimodal Data

  • Han Xuewen ,
  • Che Shangkun ,
  • Yang Mengqing ,
  • Wang Neng
Expand
  • 1 School of Economics and Management, Tsinghua University, Beijing 100084;
    2 School of Journalism and Communication, Nanjing Normal University, Nanjing 210023;
    3 Department of Electrical and Computer Engineering, University of California, Los Angeles 90095
Han Xuewen, doctoral candidate;Che Shangkun, doctoral candidate;Yang Mengqing, lecturer, PhD, correspondence author, E-mail:mqyang@nnu.edu.cn;Wang Neng, postgraduate student.

Received date: 2024-06-21

  Revised date: 2024-09-28

  Online published: 2024-12-23

Supported by

This work is supported by the general project of the National Social Science Fund of China titled “Research on Multimodal Network Data Security Situation Awareness and Risk Collaborative Governance Mechanism” (Grant No. 23BTQ076).

Fold

摘要

[目的/意义] 针对不同任务的性质设计用于多模态数据处理的AI智能体模式，有利于深入挖掘多模态数据价值，从而支持决策过程。[方法/过程] 设计一种结合水平智能体结构与垂直智能体结构的混合模式，以智能投资场景中财务年度报告为例，选择美股市场道琼斯指数的30支股票，对各个公司2023年度年报进行智能投资研究分析。利用大型语言模型处理多模态数据的能力，分析公司基本面、市场情绪与风险和机会3个子任务，并根据子任务做出最终的投资决策。[结果/结论] 垂直结构在要求工作效率和清晰决策的分析中表现出色。水平结构在需要协作、灵活性和广泛汇集知识与专业技能的场景中表现更佳。针对不同的任务选择最佳的混合智能体结构后，在最终的投资决策准确性上比单一模式的智能体结构表现更好。

关键词： 多模态数据; AI智能体; 水平智能体结构; 垂直智能体结构; 金融决策

本文引用格式

韩雪雯 , 车尚锟 , 杨梦晴 , 王能 . 多模态数据驱动的AI智能体模式设计[J]. 图书情报工作, 2024 , 68(24) : 27 -37 . DOI: 10.13266/j.issn.0252-3116.2024.24.003

Abstract

[Purpose/Significance] This study develops an AI agent model for multimodal data processing across varied tasks. By exploring the inherent value of multimodal data, the model enhances the support for decision-making processes. [Method/Process] This study designed a hybrid AI agent model that integrated both horizontal and vertical structures. Using the financial annual reports from thirty Dow Jones Industrial Average companies as a case study, this model facilitated intelligent investment research and analysis for the year 2023. The research compared the performance of different organizational structures of financial agents in executing three subtasks: analysis of company fundamentals, market sentiment, and risk and opportunity assessment. Decisions were made based on these analyses, leveraging large language models capable of processing multimodal data. [Result/Conclusion] The vertical agent structure excels in tasks requiring efficient, directional strategic analysis with clear hierarchical decision-making. Conversely, the horizontal structure is more effective in scenarios demanding collaboration, flexibility, and broad expertise. The hybrid model outperforms single-structure agents in terms of accuracy in the final investment decision, demonstrating its efficacy in optimizing task-specific agent configurations.

Key words： multi-modal data; AI agents; horizontal agent structure; vertical agent structure; financial decision

参考文献

[1] 黄永文, 孙坦, 赵瑞雪, 等. 大数据与人工智能背景下新型知识服务研究与实践[J]. 图书情报工作, 2022, 66(19): 36-46. (HUANG Y W, SUN T, ZHAO R X, et al. Research and practice of new knowledge service in the context of big data and artificial intelligence[J]. Library and information service, 2022, 66(19): 36-46.)
[2] 祁彬斌, 张文梅, 朱学芳, 等. 面向馆藏数字资源的多模交互模型与影响因素研究[J]. 图书情报工作, 2023, 67(5): 90-98. (QI B B, ZHANG W F, ZHU X F, et al. Research on multi-mode interaction model and influencing factors for digital collections[J]. Library and information service, 2023, 67(5): 90-98.)
[3] 张利锋. 基于神经网络的医疗数据处理关键技术研究[D]. 北京: 北京邮电大学, 2024. (ZHANG L F. Research on key technologies of medical data processing based on neural network[D]. Beijing: Beijing University of Posts and Telecommunications, 2024.)
[4] MAITINIYAZI M, VASIT S, PAHEDING S, et al. Soybean yield prediction from UAV using multimodal data fusion and deep learning[J]. Remote sensing of environment, 2020, 237: 111599.
[5] ZHANG Y D, DONG Z, WANG S H, et al. Advances in multimodal data fusion in neuroimaging: overview, challenges, and novel orientation[J]. Information fusion, 2020, 12(64): 149-187.
[6] 熊回香, 杨滋荣, 蒋武轩.跨媒体知识图谱构建中多模态数据语义相关性研究[J]. 情报理论与实践, 2019, 42(2): 13-18, 24. (XIONG H X, YANG Z R, JIANG W X. Semantic correlation of multimodal data in the construction of cross-media knowledge graph[J]. Information studies: theory & application, 2019, 42(2): 13-18, 24.)
[7] 李纲, 王施运, 毛进, 等.面向态势感知的国家安全事件图谱构建研究[J]. 情报学报, 2021, 40(11): 1164-1175. (LI G, WANG S Y, MAO J, et al. Construction of national security event map and its application for situation awareness[J]. Journal of the China Society for Scientific and Technical Information, 2021, 40(11): 1164-1175.)
[8] 马超, 李纲, 陈思菁, 等. 基于多模态数据语义融合的旅游在线评论有用性识别研究[J]. 情报学报, 2020, 39(2): 199-207. (MA C, LI G, CHEN S J, et al. Research on usefulness recognition of tourism online reviews based on multimodal data semantic fusion[J]. Journal of the China Society for Scientific and Technical Information, 2020, 39(2): 199-207.)
[9] SHI L, YANG X, LI J, et al. Scenario construction and deduction for railway emergency response decision-making based on network models[J]. Information sciences, 2022, 588: 331-349.
[10] FARIDA M, HAZRAT A, NADY E H, et al. Artificial intelligence-based methods for fusion of electronic health records and imaging data[J]. Scientific reports, 2022, 12(1): 17981.
[11] 吴雅威, 龚蛟腾, 朱洪涛.面向科技智库决策咨询研究需求的多元化资源融合与服务路径[J]. 图书情报工作, 2023, 67(5): 78-89. (WU Y W, GONG J T, ZHU H T. Diversified resource integration and service path for decision-making consultation research needs of scientific and technological think tanks[J]. Library and information service, 2023, 67(5): 78-89.)
[12] 王松, 焦海燕, 刘新民.融合外部知识图谱的多模态知识共创价值识别研究[J]. 情报理论与实践, 2024, 47(11): 139-149. (WANG S, JIAO H Y, LIU X M. Research on multi-modal knowledge co-creation value recognition by integrating external knowledge graph[J]. Information studies: theory & application, 2024, 47(11): 139-149.)
[13] 韩普, 陈文祺, 顾亮, 等.融合多模态数据的中文医学实体识别研究[J]. 情报理论与实践, 2024, 47(9): 174-182. (HAN P, CHEN W Q, GU L et al. Research on Chinese medical named entity recognition with fusion of multimodal data[J]. Information studies: theory & application, 2024, 47(9): 174-182.)
[14] 金光, 包阳.基于大模型的智能系统试验数据分析技术初探[J/OL]. 系统工程与电子技术:1-13[2024-05-20]. http://kns.cnki.net/kcms/detail/11.2422.TN.20240517.1849.015.html. (JIN G, BAO Y. Research on test data analysis for intelligent system using LLMs[J/OL]. Systems engineering and electronics:1-13[2024-05-20]. http://kns.cnki.net/kcms/detail/11.2422.TN.20240517.1849.015.html.)
[15] MASTERMAN T, BESEN S, SAWTELL M, et al. The Landscape of emerging AI agent architectures for reasoning, planning, and tool calling: a survey[J/OL]. arXiv preprint, 2024, arXiv: 2404.11584.
[16] 张连福, 谭作文.一种面向多模态医疗数据的联邦学习隐私保护方法[J]. 计算机科学, 2023, 50(S2): 933-940. (ZHANG L F, TAN Z W. Federated learning privacy-preserving approach for multimodal medical data[J]. Computer science, 2023, 50(S2): 933-940.)
[17] AMAL S, SAFARNEJAD L, OMIYE J A, et al. Use of multi-modal data and machine learning to improve cardiovascular disease care[J]. Frontiers in cardiovascular medicine, 2022, 9:840262.
[18] 陈孟, 干可, 李凯, 等.基于实时多模态时空数据的时空图卷积网络精准鲁棒交通流预测模型[J]. 公路交通科技, 2021, 38(8): 134-139, 158. (CHEN M, GAN K, LI K, et al. A Spatial-temporal graph convolutional network model for accurate and robust traffic flow prediction based on real-time multimodal spatial-temporal data[J]. Journal of highway and transportation research and development, 2021, 38(8): 134-139, 158.)
[19] SMARZARO R, Davis JR C A, QUINTANILHA J A. Creation of a multimodal urban transportation network through spatial data integration from authoritative and crowdsourced data[J]. ISPRS International journal of geo-information, 2021, 10(7): 470.
[20] EISELE W L, RILETT L R, BROWN MHOON, K, et al. Using intelligent transportation systems travel-time data for multimodal analyses and system monitoring[J]. Transportation research record, 2001, 1768(1): 148-156.
[21] YANG H, ZHANG B, WANG N, et al. FinRobot: An open-source AI agent platform for financial applications using large language models[J]. arXiv preprint, 2024, arXiv: 2405.14767.
[22] 李佳轩, 储节旺, 杜秀秀.关联、黑箱与赋能: AIGC驱动智慧图书馆的转型路径[J]. 图书情报工作, 2023, 67(23): 18-27. (LI J X, CHU J W, DU X X. Connection, black box and empowerment: the path of AIGC to drive the transformation of smart libraries[J]. Library and information service, 2023, 67(23): 18-27.)
[23] HIRIYANNAIAH S, SRINIVAS A M D, SHETTY G K, et al. A computationally intelligent agent for detecting fake news using generative adversarial networks[M]. Hybrid computational intelligence. San Diego: Academic Press, 2020: 69-96.
[24] YANG H. FinRobot [EB/OL]. [2024-09-17]. https://github.com/AI4Finance-Foundation/FinRobot.
[25] HUGGINGFACE. Pretrained models [EB/OL]. [2024-09-17]. https://www.sbert.net/docs/pretrained_models.html.
[26] 汪弘, 罗党论, 林东杰.行业分析师的研究报告对投资决策有用吗?——来自中国A股上市公司的经验证据[J]. 证券市场导报, 2013(7): 36-43. (WANG H, LUO D L, LIN D J. Are industry analysts’ research reports useful for investment decisions? empirical evidence from Chinese A-share listed companies[J]. Securities market herald, 2013(7): 36-43.)
[27] ABARBANELL J S, BUSHEE B J. Fundamental analysis, future earnings, and stock prices[J]. Journal of accounting research, 1997, 35(1): 1-24.
[28] BOLLEN J, MAO H, ZENG X. Twitter mood predicts the stock market [J]. Journal of computational science, 2011, 2(1): 1-8.
[29] HULL J C. The evaluation of risk in business investment[M]. Berkeley: Elsevier, 2014.
[30] HOUCINE A. The effect of financial reporting quality on corporate investment efficiency: evidence from the Tunisian stock market[J]. Research in international business and finance. 2017, 42: 321-337.
[31] ERTIMUR Y, SUNDER J, SUNDER S V. Measure for measure: the relation between forecast accuracy and recommendation profitability of analysts[J]. Journal of accounting research, 2010, 45(3): 567-606.
[32] ZHANG H, GONG J, WU W. Artificial intelligence fortext generation: an intellectual property perspective[C]// International conference on AI-generated content. Singapore: Springer Nature Singapore. 2023: 266-279.
[33] ZENG J, HUANG R, MALIK W, et al. Large language models for social networks: applications, challenges, and solutions[J]. arXiv preprint, 2024, arXiv: 2401.02575.
Options
文章导航

/

〈 〉