认知转向背景下用户相关性判断研究的方法论观察

doi:10.13266/j.issn.0252-3116.2014.18.010

图书情报工作 ›› 2014, Vol. 58 ›› Issue (18): 66-76.DOI: 10.13266/j.issn.0252-3116.2014.18.010

认知转向背景下用户相关性判断研究的方法论观察

王健¹, 王志强², 刘茜¹, 崔运鹏¹, 赵华¹, 王剑¹, 满芮¹

1. 中国农业科学院农业信息研究所;
2. 中国科学院地理科学与资源研究所

收稿日期:2014-07-09 修回日期:2014-08-14 出版日期:2014-09-20 发布日期:2014-09-20
通讯作者: 王志强，中国科学院地理科学与资源研究所助理研究员，博士，通讯作者，E-mail：wangzq@igsnrr.ac.cn
作者简介:王健，中国农业科学院农业信息研究所副研究员，博士；刘茜，中国农业科学院农业信息研究所助理研究员，硕士；崔运鹏，中国农业科学院农业信息研究所副研究员，博士；赵华，中国农业科学院农业信息研究所助理研究员，硕士；王剑，中国农业科学院农业信息研究所助理研究员，博士；满芮，中国农业科学院农业信息研究所助理研究员，硕士。
基金资助:
本文系国家社会科学基金项目“科学数据用户相关性标准与使用模式实证研究”（项目编号：14BTQ056）、国家自然科学基金项目“区域可持续发展模型集成与共享研究”（项目编号：41371536）与国家社会科学基金项目“大数据环境下的科技信息潜在语义挖掘技术优化与比较研究”（项目编号：14BTQ029）研究成果之一。

A Methodological Survey on the Study of User Relevance Judgment in Context of Cognitive Turn

Wang Jian¹, Wang Zhiqiang², Liu Qian¹, Cui Yunpeng¹, Zhao Hua¹, Wang Jian¹, Man Rui¹

1. Agricultural Information Institute, Chinese Academy of Agricultural Sciences, Beijing 100081;
2. Institute of Geography Sciences and Natural Resources Research, Chinese Acaderny of Sciences, Beijing 100039

Received:2014-07-09 Revised:2014-08-14 Online:2014-09-20 Published:2014-09-20

摘要/Abstract

摘要：

调查1967-2013年期间与用户相关性判断研究相关的82篇文献，筛选其中55篇在研究方法上具有代表性者构成样本文献。通过分析发现：样本文献在方法论思想和具体研究方法上存在较多共性和规律性的观点与做法，并可在结构上组织为一个以相关性判断的情境依赖性、认知主因和真实情境设定中开展研究3项方法原则为核心，纵向上涵盖方法论思想、研究策略和具体研究方案设计等多个层次，横向上涉及样本选取、数据采集与分析策略制定等多个研究方案设计关键环节的参考性框架。认为信息查询与检索领域的认知观是该方法框架形成、发展和进一步演化的关键驱动因素，并据此分析该框架的未来发展。

关键词: 用户相关性判断, 研究方法, 认知, 信息检索, 信息行为

Abstract:

The paper constructs a set of sample documents with 55 representative research documents from 82 published during 1967 to 2013. A survey on these sample documents shows that there are many common and regular ideas and operations among methodology thoughts and specific research methods, such methodological consensus form a reference methodological framework which 1) takes its core from three principles including situation dependency of user relevance judgment, cognitive main-factors of relevance judgment and operating research in real situation, 2) covers vertically the layers of methodology philosophy, research strategy and research planning, and 3) horizontally touches some key procedures of making a research plan such as sampling, data collecting and analyzing. The paper concludes that the cognitive view in the domain of information seeking and retrieval (ISR) is the main driver for the forming, developing and further evolving the framework. By the viewpoint the paper forecasts the future development of the framework.

Key words: user relevance judgment, research methodology, cognition, information seeking, information behavior

中图分类号:

G250

王健, 王志强, 刘茜, 崔运鹏, 赵华, 王剑, 满芮. 认知转向背景下用户相关性判断研究的方法论观察[J]. 图书情报工作, 2014, 58(18): 66-76.

Wang Jian, Wang Zhiqiang, Liu Qian, Cui Yunpeng, Zhao Hua, Wang Jian, Man Rui. A Methodological Survey on the Study of User Relevance Judgment in Context of Cognitive Turn[J]. LIS, 2014, 58(18): 66-76.

参考文献

[1] Ingwersen P, Jarvelin K. The turn: Integration of information seeking and retrieval in context(the information retrieval series)[M]. New York: Springer-Verlag, Inc., 2005.

[2] Robertson S E, Hancock-Beaulieu M M.On the evaluation of IR systems[J]. Information Processing & Management, 1992, 28(4): 457-466.

[3] Cuadra C A, Katter R V. Opening the black box of “relevance”[J]. Journal of Documentation, 1967, 23(4): 291-303.

[4] Rees A M, Schultz D G. A field experimental approach to the study of relevance assessments in relation to document searching[R]. Cleveland: Case Western Reserve University,1967.

[5] Schamber L. Users' criteria for evaluation in multimedia information seeking and use situations[D]. New York: Syracuse University, 1991.

[6] Park T K.The nature of relevance in information retrieval: An empirical study[J]. The Library Quarterly, 1993,63(3):318-351.

[7] Cuadra C A.Experimental studies of relevance judgments. Final report. Volume 1:Project summary[M].Santa Monica: System Development Corporation, 1967.

[8] Bateman J A. Modeling changes in end-user relevance criteria: An information seeking study[D]. Texas: University of North Texas, 1998.

[9] Vakkari P, Hakala N. Changes in relevance criteria and problem stages in task performance[J]. Journal of Documentation, 2000, 56(5): 540-562.

[10] Eisenberg M, Barry C. Order effects: A study of the possible influence of presentation order on user judgments of document relevance[J]. Journal of the American Society for Information Science, 1988, 39(5): 293-300.

[11] Janes J W, McKinney R. Relevance judgments of actual users and secondary judges: A comparative study[J]. The Library Quarterly, 1992, 62(2): 150-168.

[12] Janes J W. Other people's judgments:A comparison of users' and others' judgments of document relevance, topicality, and utility[J]. Journal of the American Society for Information Science, 1994, 45(3): 160-171.

[13] Spink A, Greisdorf H, Bateman J. Examining different regions of relevance: From highly relevant to not relevant[C]//Proceedings of the ASIS Annual Meeting. Pittsburgh: American Society for Information Science and Technology, 1998: 3-12.

[14] Hirsh S G. Children's relevance criteria and information seeking on electronic resources[J]. Journal of the American Society for Information Science, 1999, 50(14): 1265-1283.

[15] Cool C, Belkin N, Frieder O, et al. Characteristics of text affecting relevance judgments[C]//Proceedings of the 14th National Online Meeting. New York: Learned Information Inc., 1993: 77-81.

[16] Janes J W. Relevance judgments and the incremental presentation of document representations[J]. Information Processing & Management, 1991, 27(6): 629-646.

[17] Maglaughlin K L, Sonnenwald D H. User perspectives on relevance criteria: A comparison among relevant, partially relevant, and not-relevant judgments[J]. Journal of the American Society for Information Science and Technology, 2002, 53(5): 327-342.

[18] Sedghi S, Sanderson M, Clough P. A study on the relevance criteria for medical images[J]. Pattern Recognition Letters, 2008, 29(15): 2046-2057.

[19] Savolainen R, Kari J. User-defined relevance criteria in Web searching[J]. Journal of Documentation, 2006, 62(6): 685-707.

[20] Taylor A R, Cool C, Belkin N J,et al. Relationships between categories of relevance criteria and stage in task completion[J]. Information Processing & Management, 2007, 43(4): 1071- 1084.

[21] Taylor A. Relevance criterion choices in relation to search progress[D]. New Brunswick: Rutgers University, 2009.

[22] Taylor A, Zhang Xiangmin, Amadio W J. Examination of relevance criteria choices and the information search process[J]. Journal of Documentation, 2009, 65(5): 719-744.

[23] Papaeconomou C, Zijlema A F, Ingwersen P. Searchers' relevance judgments and criteria in evaluating Web pages in a learning style perspective[C]//Proceedings of the Second International Symposium on Information Interaction in Context. Texas: ACM, 2008: 123-132.

[24] Burton V T, Chadwick S A. Investigating the practices of student researchers: Patterns of use and criteria for use of Internet and library sources[J]. Computers and Composition, 2000, 17(3): 309-328.

[25] Hamid R A, Thom J A. Criteria that have an effect on users while making image relevance judgments[C]//Fifteenth Australasian Document Computing Symposium. Melbourne:RMIT University, 2010: 1-8.

[26] Balatsoukas P, Ruthven I. What eyes can tell about the use of relevance criteria during predictive relevance judgment?[C]//Proceedings of the Third Symposium on Information Interaction in Context. Texas: ACM, 2010: 389-394.

[27] Balatsoukas P, Ruthven I. An eye-tracking approach to the analysis of relevance judgments on the Web: The case of Google search engine[J]. Journal of the American Society for Information Science and Technology, 2012, 63(9): 1728-1746.

[28] Balatsoukas P, Ruthven I. The use of relevance criteria during predictive judgment: An eye tracking approach[J]. Proceedings of the American Society for Information Science and Technology, 2010, 47(1): 1-10.

[29] Inskip C, MacFarlane A, Rafferty P. Creative professional users' musical relevance criteria[J]. Journal of Information Science, 2010, 36(4): 517-529.

[30] Savolainen R. Source preference criteria in the context of everyday projects: Relevance judgments made by prospective home buyers[J]. Journal of Documentation, 2010, 66(1): 70-92.

[31] Laplante A. Users' relevance criteria in music retrieval in everyday life: An exploratory study[C]//Proceedings of the 11th International Society for Music Information Retrieval Conference. Utrecht: ISMIR, 2010: 601-606.

[32] Taylor A. User relevance criteria choices and the information search process[J]. Information Processing & Management, 2012, 48(1): 136-153.

[33] Taylor A. Examination of work task and criteria choices for the relevance judgment process[J]. Journal of Documentation, 2013, 69(4): 523-544.

[34] Janes J W. The binary nature of continuous relevance judgments: A study of users' perceptions[J]. Journal of the American Society for Information Science, 1991, 42(10): 754-756.

[35] Wang Peiling. Contextualizing user relevance criteria:A meta-ethnographic approach to user-centered relevance studies[C]//Proceedings of the Third Symposium on Information Interaction in Context. Texas: ACM, 2010: 293-298.

[36] Wang Peiling. A cognitive model of document selection of real users of information retrieval systems[M]. College Park: University of Maryland at College Park, 1994.

[37] Tang Rong, Solomon P. Toward an understanding of the dynamics of relevance judgment: An analysis of one person's search behavior[J]. Information Processing & Management, 1998, 34(2): 237-256.

[38] Wang Peiling, Soergel D. A cognitive model of document use during a research project. Study I. Document selection[J]. Journal of the American Society for Information Science, 1998, 49(2): 115-133.

[39] Wang Peiling, Domas White M. A cognitive model of document use during a research project. Study ll. Decisions at the reading and citing stages[J]. Journal of the American Society for Information Science, 1999, 50(2): 98-114.

[40] Howard D L. Pertinence as reflected in personal constructs[J]. Journal of the American Society for Information Science, 1994, 45(3): 172-185.

[41] Tang Rong, Solomon P. Use of relevance criteria across stages of document evaluation: On the complementarity of experimental and naturalistic studies[J]. Journal of the American Society for information Science and Technology, 2001, 52(8): 676-685.

[42] Kim S, Oh J S, Oh S. Best-answer selection criteria in a social Q&A site from the user-oriented relevance perspective[J]. Proceedings of the American Society for Information Science and Technology, 2007, 44(1): 1-15.

[43] 成颖. 信息检索相关性判据及应用研究[D]. 南京: 南京大学, 2011.

[44] Barry C L. The identification of user criteria of relevance and document characteristics: Beyond the topical approach to information retrieval[D]. New York: Syracuse University, 1993.

[45] Park T K. The nature of relevance in information retrieval: An empirical study[J]. The Library Quarterly, 1993, 63(3): 318-351.

[46] Meng Yang, Marchionini G. Exploring users' video relevance criteria-A pilot study[C]//Proceedings of the American Society for Information Science and Technology. Providence: American Society for Information Science and Technology, 2004: 229-238.

[47] Barry C L. User-defined relevance criteria:An exploratory study[J]. Journal of American Society and Information Science, 1994, 45(3): 149-159.

[48] Nilan M S, Peek R P, Snyder H W. A methodology for tapping user evaluation behaviors: An exploration of users' strategy, source and information evaluating[C]//Proceedings of the 51st Annual Meeting of the American Society for Information Science. Atlanta: American Society for Information Science and Technology,1988:152-159.

[49] Crystal A, Greenberg J. Relevance criteria identified by health information users during Web searches[J]. Journal of the American Society for information Science and Technology, 2006, 57(10): 1368-1382.

[50] Lawley K N, Soergel D, Huang Xiaoli. Relevance criteria used by teachers in selecting oral history materials[J]. Proceedings of the American Society for Information Science and Technology, 2005, 42(1): 421-448.

[51] Westbrook L. Faculty relevance criteria: Internalized user needs[J]. Library Trends, 2001, 50(2): 197-206.

[52] Park T K. Toward a theory of user-based relevance: A call for a new paradigm of inquiry[J]. Journal of the American Society for Information Science, 1994, 45(3): 135-141.

[53] Schamber L, Bateman J. User criteria in relevance evaluation: Toward development of a measurement scale[C]//Proceedings of the Annual Meeting-American Society for Information Science. Baltirnore: American Society for Information Science and Technology, 1996: 218-225.

[54] Tombros A, Ruthven I, Jose J M. Searchers' criteria for assessing Web pages[C]//Proceedings of the 26th Annual International ACM SIGIR Conference on Research and Development in Informaion Retrieval. Texas: ACM, 2003: 385-386.

[55] 于春, 彭爱东, 王波,等. 信息用户对信息检索相关性判断的因素分析[J]. 图书情报工作, 2009，53(3): 103-107.

[56] Barnes M D, Penrod C, Neiger B L,et al. Measuring the relevance of evaluation criteria among health information seekers on the Internet[J]. Journal of Health Psychology, 2003, 8(1): 71-82.

[57] Goodrum A, Pope R, Godo E, et al. Newsblog relevance: Applying relevance criteria to news-related blogs[J]. Proceedings of the American Society for Information Science and Technology, 2010, 47(1): 1-2.

[58] 童迎, 李鹏, 夏慧. 合作信息查寻与检索相关性判据的探索性因子分析[J]. 图书情报工作, 2013, 57(19): 30-36.

[59] Schamber L. Time-line interviews and inductive content analysis:Their effectiveness for exploring cognitive behaviors[J]. Journal of the American Society for Information Science, 2000, 51(8): 734-744.

[60] Case D O. Looking for information: A survey of research on information seeking, needs and behavior[M]. Bingley: Emerald Group Publishing, 2012.

认知转向背景下用户相关性判断研究的方法论观察

A Methodological Survey on the Study of User Relevance Judgment in Context of Cognitive Turn

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

编辑推荐

Metrics

[1]	孙悦, 黄微. 社交媒体平台用户参与的行为谱与行为层级模型构建[J]. 图书情报工作, 2022, 66(9): 40-52.
[2]	王志红, 曹树金. 文本信息检索中相关性判断的次序效应及其情境因素研究[J]. 图书情报工作, 2022, 66(9): 128-140.
[3]	王馨悦, 刘畅. 时间限制与时间压力下的信息行为研究综述[J]. 图书情报工作, 2022, 66(9): 141-151.
[4]	谢雨杉, 柯青, 王笑语, 秦琴. 新冠疫情背景下情绪与信息行为的关系及情绪角色的主题分析[J]. 图书情报工作, 2022, 66(8): 102-112.
[5]	黄崑, 陈佳琦, 刘畅. 面向任务难度感知的检索行为研究[J]. 图书情报工作, 2022, 66(3): 83-92.
[6]	王今, 马海群, 邹纯龙. 国家情报评估与决策的关系——基于多源流理论和认知心理学双重视角[J]. 图书情报工作, 2022, 66(2): 22-31.
[7]	樊振佳, 骆卓昱. 高校学生“数字脱瘾”行为影响因素研究[J]. 图书情报工作, 2022, 66(17): 106-115.
[8]	朱强, 卢文辉, 吴亚平, 李晓东, 别立谦, 叶继元. 新时代面向信息资源保障的用户信息需求与信息行为调查研究[J]. 图书情报工作, 2022, 66(15): 23-33.
[9]	李洪晨, 马捷, 高鹤萌, 郝志远. 面向发展性阅读障碍的数字阅读与纸质阅读比较研究[J]. 图书情报工作, 2022, 66(11): 66-75.
[10]	卢文辉. 学位论文引文视角下的硕士生学术文献信息资源使用行为特征研究[J]. 图书情报工作, 2022, 66(10): 131-142.
[11]	黄国彬, 毕子晗, 孙静彤, 付映宏. 科研人员对注册式研究报告的认知水平及其提升策略研究——基于北京8所“双一流”高校的调研[J]. 图书情报工作, 2022, 66(1): 89-95.
[12]	孙海霞. 国外健康信息规避行为研究综述[J]. 图书情报工作, 2021, 65(9): 138-150.
[13]	肖鹏, 衣春波. 万华镜里的自画像:关于中国人文学者数字学术认知与需求的质性研究[J]. 图书情报工作, 2021, 65(7): 4-14.
[14]	刘坤锋, 李艳红, 张心源. 信任视角下数据智能与专家知识的对比实证研究[J]. 图书情报工作, 2021, 65(6): 110-117.
[15]	刘萍, 王朝阳, 倪江雪. 大学生网络协作学习中的认知策略研究[J]. 图书情报工作, 2021, 65(3): 109-117.