收稿日期: 2014-07-03
修回日期: 2014-07-09
网络出版日期: 2014-08-20
基金资助
本文系国家社会科学基金项目“基于用户多维交互行为的数字图书馆可持续发展评估模型与实证研究”（项目编号：11BTQ009）研究成果之一。
Task Design in Information Retrieval Experiment：A Comparative Study on Real and Simulated Work Tasks
Received date: 2014-07-03
Revised date: 2014-07-09
Online published: 2014-08-20
在信息检索系统评估和信息搜寻行为研究中，实验法是常用的研究方法，而实验中的任务设计是实验成功的关键环节。设计模拟仿真工作任务情境是近年来常用的任务设计方式，而它能否替代用户的真实工作任务在很大程度上影响着实验研究结果的科学性。为验证此方法，招募42位本科生参加实验，要求其为完成一项真实工作任务和一项仿真工作任务，在CNKI中展开搜索。通过分析实验参加者在实施两项搜索任务中的交互信息检索行为和检索绩效，发现仿真工作任务与真实工作任务在属性上尽管存在某些显著差异，但并未对用户交互信息搜索行为及绩效产生显著影响。因而，仿真工作任务可以替代真实工作任务用于交互信息检索实验研究，但需要进行合理设计。
李月琳 , 肖雪 , 胡蝶 . 信息检索实验中的任务设计——真实与模拟仿真工作任务的比较研究[J]. 图书情报工作, 2014 , 58(16) : 5 -12 . DOI: 10.13266/j.issn.0252-3116.2014.16.001
Experiment is a widely used research method in information retrieval systems evaluation and information seeking behavior research. Task design is critical in the experiment because it significantly affects research results. Simulated work task situations are used frequently in experiments in recent years. However, whether it could take place of users' real work tasks affects the results of experiments to a great degree. This study conducts an experiment which recruits 42 undergraduate students. A simulated work task situation is designed and assigned to the participants to search in CNKI, who were asked to bring a real work task situation to the experiment. The paper analyzes characteristics of the two tasks in interactive information retrieval behavior and retrieval performance. Some significant differences in properties are recognized, but there are no significant effect found in users' behaviors and interactive performance when they engaged in two tasks. The paper concludes that simulated work tasks can replace real work tasks in experiment, but it needs to be carefully designed. Some related suggestions are discussed.
[1] Borlund P, Ingwersen P. The development of a method for the evaluation of interactive information retrieval systems[J]. Journal of Documentation, 1997,53(3):225-250.
[2] Borlund P. Experimental components for the evaluation of interactive information retrieval systems[J]. Journal of Documentation, 2000,56(1): 71-90.
[3] Borlund P, Schneider J W. Reconsideration of the simulated work task situation: A context instrument for evaluation of information retrieval interaction[EB/OL].[2013-09-16]. http://pure.iva.dk/files/31012378/Borlund%2520Schneider%2520IIiX%25202010.pdf.
[4] Blomgren L, Vallo H, Byström K. Evaluation of an information system in an information seeking process[M]//Heery R, Lyon L. Research and Advanced Technology for Digital Libraries. Berlin: Springer, 2004:57-68.
[5] Vakkari P. Task-based information searching[J]. Annual Review of Information Science and Technology, 2003,37(1):413-464.
[6] Kim J. Describing and predicting information seeking behavior on the Web[J]. Journal of the American Society for Information Science and Technology, 2009,60(4):679-693.
[7] Li Yuelin, Belkin N J. An exploration of the relationships between work task and interactive information search behavior[J]. Journal of the American Society for Information Science and Technology, 2010,61(9):1771-1789.
[8] Byström K, Järvelin K. Task complexity affects information seeking and use[J]. Information Processing & Management, 1995,31(2):191-213.
[9] Bell D J, Ruthven I. Searcher's assessments of task complexity for Web searching[C]//McDonald S, Tait J. Advances in Information Retrieval: 26th European Conference on IR Research. Berlin: Springer, 2004:57-71.
[10] Gwizdka J, Spence I. What can searching behavior tell us about the difficulty of information tasks? A study of Web navigation[J]. Proceedings of the American Society for Information Science and Technology,2006,43(1):1-22.
[11] Li Yuelin, Belkin N J. A faceted approach to conceptualizing tasks in information seeking[J]. Information Processing & Management, 2008,44(6):1822-1837.
[12] Li Yuelin. Exploring the relationships between work task and search task in information search[J]. Journal of the American Society for Information Science and Technology, 2009,60(2):275-291.
[13] Liu Jingjing,Belkin N J. Personalizing information retrieval for multi-session tasks: The roles of task stage and task type[C]//Proceedings of the 33rd International ACM SIGIR Conference on Research and Development in Information Retrieval.New York:ACM, 2010: 26-33.
[14] Liu Jingjing,Gwizdka J, Liu Chang, et al. Predicting task difficulty for different task types[J]. Proceedings of the American Society for Information Science and Technology, 2010,47(1):1-10.
[15] Xie Iris. Dimensions of tasks: Influences on information-seeking and retrieving processes[J]. Journal of Documentation,2009,65(3):339-366.
[16] Li Yuelin. Investigating the relationships between facets of work task and selection and query-related behavior[J]. Chinese Journal of Library and Information Science, 2012,5(1):51-69.
[17] Yuan Xiaojun, Belkin N J. Investigating information retrieval support techniques for different information-seeking strategies[J]. Journal of the American Society for Information Science and Technology, 2010,61(8):1543-1563.
[18] Yuan Xiaojun, Belkin N J. Evaluating an integrated system supporting multiple information-seeking strategies[J]. Journal of the American Society for Information Science and Technology, 2010,61(10):1987-2010.
[19] Maynard D C,Hakel M D. Effects of objective and subjective task complexity on performance[J]. Human Performance,1997,10(4):303-330.
[20] Liu Jingjing, Liu Chang, Yuan Xiaojun, et al. Understanding searchers' perception of task difficulty:Relationships with task type[J]. Proceedings of the American Society for Information Science and Technology,2011,48(1):1-10.
[21] Li Yuelin. An exploration of the relationships between work tasks and users' interaction performance[J]. Proceedings of the American Society for Information Science and Technology,2010,47(1):1-9.
/
| 〈 |
|
〉 |
