收稿日期: 2014-07-09
修回日期: 2014-08-14
网络出版日期: 2014-09-20
基金资助
本文系国家社会科学基金项目“科学数据用户相关性标准与使用模式实证研究”（项目编号：14BTQ056）、国家自然科学基金项目“区域可持续发展模型集成与共享研究”（项目编号：41371536）与国家社会科学基金项目“大数据环境下的科技信息潜在语义挖掘技术优化与比较研究”（项目编号：14BTQ029）研究成果之一。
A Methodological Survey on the Study of User Relevance Judgment in Context of Cognitive Turn
Received date: 2014-07-09
Revised date: 2014-08-14
Online published: 2014-09-20
调查1967-2013年期间与用户相关性判断研究相关的82篇文献，筛选其中55篇在研究方法上具有代表性者构成样本文献。通过分析发现：样本文献在方法论思想和具体研究方法上存在较多共性和规律性的观点与做法，并可在结构上组织为一个以相关性判断的情境依赖性、认知主因和真实情境设定中开展研究3项方法原则为核心，纵向上涵盖方法论思想、研究策略和具体研究方案设计等多个层次，横向上涉及样本选取、数据采集与分析策略制定等多个研究方案设计关键环节的参考性框架。认为信息查询与检索领域的认知观是该方法框架形成、发展和进一步演化的关键驱动因素，并据此分析该框架的未来发展。
王健 , 王志强 , 刘茜 , 崔运鹏 , 赵华 , 王剑 , 满芮 . 认知转向背景下用户相关性判断研究的方法论观察[J]. 图书情报工作, 2014 , 58(18) : 66 -76 . DOI: 10.13266/j.issn.0252-3116.2014.18.010
The paper constructs a set of sample documents with 55 representative research documents from 82 published during 1967 to 2013. A survey on these sample documents shows that there are many common and regular ideas and operations among methodology thoughts and specific research methods, such methodological consensus form a reference methodological framework which 1) takes its core from three principles including situation dependency of user relevance judgment, cognitive main-factors of relevance judgment and operating research in real situation, 2) covers vertically the layers of methodology philosophy, research strategy and research planning, and 3) horizontally touches some key procedures of making a research plan such as sampling, data collecting and analyzing. The paper concludes that the cognitive view in the domain of information seeking and retrieval (ISR) is the main driver for the forming, developing and further evolving the framework. By the viewpoint the paper forecasts the future development of the framework.
[1] Ingwersen P, Jarvelin K. The turn: Integration of information seeking and retrieval in context(the information retrieval series)[M]. New York: Springer-Verlag, Inc., 2005.
[2] Robertson S E, Hancock-Beaulieu M M.On the evaluation of IR systems[J]. Information Processing & Management, 1992, 28(4): 457-466.
[3] Cuadra C A, Katter R V. Opening the black box of “relevance”[J]. Journal of Documentation, 1967, 23(4): 291-303.
[4] Rees A M, Schultz D G. A field experimental approach to the study of relevance assessments in relation to document searching[R]. Cleveland: Case Western Reserve University,1967.
[5] Schamber L. Users' criteria for evaluation in multimedia information seeking and use situations[D]. New York: Syracuse University, 1991.
[6] Park T K.The nature of relevance in information retrieval: An empirical study[J]. The Library Quarterly, 1993,63(3):318-351.
[7] Cuadra C A.Experimental studies of relevance judgments. Final report. Volume 1:Project summary[M].Santa Monica: System Development Corporation, 1967.
[8] Bateman J A. Modeling changes in end-user relevance criteria: An information seeking study[D]. Texas: University of North Texas, 1998.
[9] Vakkari P, Hakala N. Changes in relevance criteria and problem stages in task performance[J]. Journal of Documentation, 2000, 56(5): 540-562.
[10] Eisenberg M, Barry C. Order effects: A study of the possible influence of presentation order on user judgments of document relevance[J]. Journal of the American Society for Information Science, 1988, 39(5): 293-300.
[11] Janes J W, McKinney R. Relevance judgments of actual users and secondary judges: A comparative study[J]. The Library Quarterly, 1992, 62(2): 150-168.
[12] Janes J W. Other people's judgments:A comparison of users' and others' judgments of document relevance, topicality, and utility[J]. Journal of the American Society for Information Science, 1994, 45(3): 160-171.
[13] Spink A, Greisdorf H, Bateman J. Examining different regions of relevance: From highly relevant to not relevant[C]//Proceedings of the ASIS Annual Meeting. Pittsburgh: American Society for Information Science and Technology, 1998: 3-12.
[14] Hirsh S G. Children's relevance criteria and information seeking on electronic resources[J]. Journal of the American Society for Information Science, 1999, 50(14): 1265-1283.
[15] Cool C, Belkin N, Frieder O, et al. Characteristics of text affecting relevance judgments[C]//Proceedings of the 14th National Online Meeting. New York: Learned Information Inc., 1993: 77-81.
[16] Janes J W. Relevance judgments and the incremental presentation of document representations[J]. Information Processing & Management, 1991, 27(6): 629-646.
[17] Maglaughlin K L, Sonnenwald D H. User perspectives on relevance criteria: A comparison among relevant, partially relevant, and not-relevant judgments[J]. Journal of the American Society for Information Science and Technology, 2002, 53(5): 327-342.
[18] Sedghi S, Sanderson M, Clough P. A study on the relevance criteria for medical images[J]. Pattern Recognition Letters, 2008, 29(15): 2046-2057.
[19] Savolainen R, Kari J. User-defined relevance criteria in Web searching[J]. Journal of Documentation, 2006, 62(6): 685-707.
[20] Taylor A R, Cool C, Belkin N J,et al. Relationships between categories of relevance criteria and stage in task completion[J]. Information Processing & Management, 2007, 43(4): 1071- 1084.
[21] Taylor A. Relevance criterion choices in relation to search progress[D]. New Brunswick: Rutgers University, 2009.
[22] Taylor A, Zhang Xiangmin, Amadio W J. Examination of relevance criteria choices and the information search process[J]. Journal of Documentation, 2009, 65(5): 719-744.
[23] Papaeconomou C, Zijlema A F, Ingwersen P. Searchers' relevance judgments and criteria in evaluating Web pages in a learning style perspective[C]//Proceedings of the Second International Symposium on Information Interaction in Context. Texas: ACM, 2008: 123-132.
[24] Burton V T, Chadwick S A. Investigating the practices of student researchers: Patterns of use and criteria for use of Internet and library sources[J]. Computers and Composition, 2000, 17(3): 309-328.
[25] Hamid R A, Thom J A. Criteria that have an effect on users while making image relevance judgments[C]//Fifteenth Australasian Document Computing Symposium. Melbourne:RMIT University, 2010: 1-8.
[26] Balatsoukas P, Ruthven I. What eyes can tell about the use of relevance criteria during predictive relevance judgment?[C]//Proceedings of the Third Symposium on Information Interaction in Context. Texas: ACM, 2010: 389-394.
[27] Balatsoukas P, Ruthven I. An eye-tracking approach to the analysis of relevance judgments on the Web: The case of Google search engine[J]. Journal of the American Society for Information Science and Technology, 2012, 63(9): 1728-1746.
[28] Balatsoukas P, Ruthven I. The use of relevance criteria during predictive judgment: An eye tracking approach[J]. Proceedings of the American Society for Information Science and Technology, 2010, 47(1): 1-10.
[29] Inskip C, MacFarlane A, Rafferty P. Creative professional users' musical relevance criteria[J]. Journal of Information Science, 2010, 36(4): 517-529.
[30] Savolainen R. Source preference criteria in the context of everyday projects: Relevance judgments made by prospective home buyers[J]. Journal of Documentation, 2010, 66(1): 70-92.
[31] Laplante A. Users' relevance criteria in music retrieval in everyday life: An exploratory study[C]//Proceedings of the 11th International Society for Music Information Retrieval Conference. Utrecht: ISMIR, 2010: 601-606.
[32] Taylor A. User relevance criteria choices and the information search process[J]. Information Processing & Management, 2012, 48(1): 136-153.
[33] Taylor A. Examination of work task and criteria choices for the relevance judgment process[J]. Journal of Documentation, 2013, 69(4): 523-544.
[34] Janes J W. The binary nature of continuous relevance judgments: A study of users' perceptions[J]. Journal of the American Society for Information Science, 1991, 42(10): 754-756.
[35] Wang Peiling. Contextualizing user relevance criteria:A meta-ethnographic approach to user-centered relevance studies[C]//Proceedings of the Third Symposium on Information Interaction in Context. Texas: ACM, 2010: 293-298.
[36] Wang Peiling. A cognitive model of document selection of real users of information retrieval systems[M]. College Park: University of Maryland at College Park, 1994.
[37] Tang Rong, Solomon P. Toward an understanding of the dynamics of relevance judgment: An analysis of one person's search behavior[J]. Information Processing & Management, 1998, 34(2): 237-256.
[38] Wang Peiling, Soergel D. A cognitive model of document use during a research project. Study I. Document selection[J]. Journal of the American Society for Information Science, 1998, 49(2): 115-133.
[39] Wang Peiling, Domas White M. A cognitive model of document use during a research project. Study ll. Decisions at the reading and citing stages[J]. Journal of the American Society for Information Science, 1999, 50(2): 98-114.
[40] Howard D L. Pertinence as reflected in personal constructs[J]. Journal of the American Society for Information Science, 1994, 45(3): 172-185.
[41] Tang Rong, Solomon P. Use of relevance criteria across stages of document evaluation: On the complementarity of experimental and naturalistic studies[J]. Journal of the American Society for information Science and Technology, 2001, 52(8): 676-685.
[42] Kim S, Oh J S, Oh S. Best-answer selection criteria in a social Q&A site from the user-oriented relevance perspective[J]. Proceedings of the American Society for Information Science and Technology, 2007, 44(1): 1-15.
[43] 成颖. 信息检索相关性判据及应用研究[D]. 南京: 南京大学, 2011.
[44] Barry C L. The identification of user criteria of relevance and document characteristics: Beyond the topical approach to information retrieval[D]. New York: Syracuse University, 1993.
[45] Park T K. The nature of relevance in information retrieval: An empirical study[J]. The Library Quarterly, 1993, 63(3): 318-351.
[46] Meng Yang, Marchionini G. Exploring users' video relevance criteria-A pilot study[C]//Proceedings of the American Society for Information Science and Technology. Providence: American Society for Information Science and Technology, 2004: 229-238.
[47] Barry C L. User-defined relevance criteria:An exploratory study[J]. Journal of American Society and Information Science, 1994, 45(3): 149-159.
[48] Nilan M S, Peek R P, Snyder H W. A methodology for tapping user evaluation behaviors: An exploration of users' strategy, source and information evaluating[C]//Proceedings of the 51st Annual Meeting of the American Society for Information Science. Atlanta: American Society for Information Science and Technology,1988:152-159.
[49] Crystal A, Greenberg J. Relevance criteria identified by health information users during Web searches[J]. Journal of the American Society for information Science and Technology, 2006, 57(10): 1368-1382.
[50] Lawley K N, Soergel D, Huang Xiaoli. Relevance criteria used by teachers in selecting oral history materials[J]. Proceedings of the American Society for Information Science and Technology, 2005, 42(1): 421-448.
[51] Westbrook L. Faculty relevance criteria: Internalized user needs[J]. Library Trends, 2001, 50(2): 197-206.
[52] Park T K. Toward a theory of user-based relevance: A call for a new paradigm of inquiry[J]. Journal of the American Society for Information Science, 1994, 45(3): 135-141.
[53] Schamber L, Bateman J. User criteria in relevance evaluation: Toward development of a measurement scale[C]//Proceedings of the Annual Meeting-American Society for Information Science. Baltirnore: American Society for Information Science and Technology, 1996: 218-225.
[54] Tombros A, Ruthven I, Jose J M. Searchers' criteria for assessing Web pages[C]//Proceedings of the 26th Annual International ACM SIGIR Conference on Research and Development in Informaion Retrieval. Texas: ACM, 2003: 385-386.
[55] 于春, 彭爱东, 王波,等. 信息用户对信息检索相关性判断的因素分析[J]. 图书情报工作, 2009，53(3): 103-107.
[56] Barnes M D, Penrod C, Neiger B L,et al. Measuring the relevance of evaluation criteria among health information seekers on the Internet[J]. Journal of Health Psychology, 2003, 8(1): 71-82.
[57] Goodrum A, Pope R, Godo E, et al. Newsblog relevance: Applying relevance criteria to news-related blogs[J]. Proceedings of the American Society for Information Science and Technology, 2010, 47(1): 1-2.
[58] 童迎, 李鹏, 夏慧. 合作信息查寻与检索相关性判据的探索性因子分析[J]. 图书情报工作, 2013, 57(19): 30-36.
[59] Schamber L. Time-line interviews and inductive content analysis:Their effectiveness for exploring cognitive behaviors[J]. Journal of the American Society for Information Science, 2000, 51(8): 734-744.
[60] Case D O. Looking for information: A survey of research on information seeking, needs and behavior[M]. Bingley: Emerald Group Publishing, 2012.
/
| 〈 |
|
〉 |
