图书情报工作 ›› 2022, Vol. 66 ›› Issue (9): 128-140.DOI: 10.13266/j.issn.0252-3116.2022.09.013

• 知识组织 • 上一篇    下一篇

文本信息检索中相关性判断的次序效应及其情境因素研究

王志红1,2, 曹树金3   

  1. 1. 清华大学计算机科学与技术系, 北京 100084;
    2. 北京信息科学与技术国家研究中心, 北京 100084;
    3. 中山大学信息管理学院, 广州 510006
  • 收稿日期:2021-11-23 修回日期:2022-02-16 出版日期:2022-05-05 发布日期:2022-05-12
  • 通讯作者: 曹树金,教授,博士,博士生导师,通信作者,E-mail:caosj@mail.sysu.edu.cn
  • 作者简介:王志红,博士后
  • 基金资助:
    本文系国家社会科学基金重大项目"基于特定领域的网络资源知识组织与导航机制研究"(项目编号:12&ZD222)和中国博士后面上资助项目"复杂任务下基于用户相关性反馈的会话搜索优化研究"(项目编号:2021M691823)研究成果之一。

Research on Order Effect in Relevance Judgment and Its Contextual Factors in Textual Information Retrieval

Wang Zhihong1,2, Cao Shujin3   

  1. 1. Department of Computer Science and Technology, Tsinghua University, Beijing 100084;
    2. Beijing National Research Center for Information Science and Technology, Beijing 100084;
    3. School of Information Management, Sun Yat-sen University, Guangzhou 510006
  • Received:2021-11-23 Revised:2022-02-16 Online:2022-05-05 Published:2022-05-12

摘要: [目的/意义]基于相关性分值对搜索结果进行列表式线性排列是目前大多数信息检索系统采用的主要方式。已有研究虽然从理论上批判了这种方式的局限性,指出先后检验和阅读的文献之间是相互影响的,但是并未有实证研究进一步调查相关性判断次序效应发生的情境因素。[方法/过程]针对这一问题,采用2X2X2被试间实验法,招募80名被试,要求每人完成4个主题的实验任务,在实验任务完成前后通过问卷调查等方法收集数据,并采用卡方检验、全概率公式背反以及QQ等式等多种方法进行数据分析,以探究相关性判断的次序效应以及文献间关系、文献呈现形式作为情境因素所产生的影响。[结果/结论]研究结果表明,文献呈现次序会影响相关性判断,并且发现这种效应会根据文献间关系以及文献呈现形式的不同而不同。具体而言,研究发现相对于具有互补关系的文献,具有互斥关系的文献的呈现次序更容易影响相关性判断;另外,相对于文献全文,文献片段的呈现次序更容易影响相关性判断。本研究虽然存在一定的局限性,但是研究结论对于文本信息检索的理论、实践以及研究方法均具有一定的启示。

关键词: 相关性判断, 次序效应, 文献间关系, 文献呈现形式, 量子认知

Abstract: [Purpose/Significance] The main approach adopted in most information retrieval systems currently is to present the search results list linearly based on the relevance score. Although existing researches have theoretically criticized the limitations of this approach, proposing that there is mutual influence between the document previously examined and read, there is no empirical study to further investigate the contextual factors affecting the order effect of relevance judgment. [Method/Process] In response to this question, this paper adopted the 2X2X2 inter-subject experiment method by recruiting 80 subjects. Each subject was required to complete 4 experimental tasks with different topics. Data was collected by questionnaires before and after each experimental task was completed. In order to explore the order effect of relevance judgment and the influence of inter-document relationships as well as the document presentation types as contextual factors, the collected data was analyzed using various methods, such as chi-square test, law of total probability violation, QQ equation, etc. [Result/Conclusion] The research results confirm that the judgment of relevance can be affected by the presentation order of the documents and this effect is different according to the relationships between documents and document presentation types. Specifically, compared to documents with complementary relationships, the order of documents with contradictory relationship is more likely to influence the relevance judgment. In addition, compared with the full text, the order of the presentation of literature snippets is more likely to affect the relevance judgment. Although this paper has its limitations, the research conclusions obtained have certain theoretical, practical and methodological enlightenment.

Key words: relevance judgment, order effect, inter-document relationship, document presentation type, quantum cognition

中图分类号: