[Purpose/significance] Information search extensively influence people's daily lives, and people tend to use search engine to find the information they need. However, the existing search system is better for supporting factual tasks search, and it is not satisfactory to support tasks such as searching in order to learn. In recent years, researchers have begun to pay attention to regarding search as learning, and try to evaluate the knowledge learning in search.[Method/process] For the purpose of mastering users' learning status in search thoroughly, in this paper, 32 students from Peking University were recruited to complete user experiment. User knowledge assessment method considering knowledge quantity and quality comprehensively was established in this paper. Based on this knowledge assessment method, user knowledge level in learning tasks before and after search was evaluated respectively, and the changes of knowledge level were investigated.[Result/conclusion]The research shows that users' performance on knowledge quantity dimension becomes more comprehensive and in-depth with the search completion, and has a significant improvement in the number of knowledge points, the number of knowledge facets, the breadth of knowledge facets and the depth of knowledge facets. At the same time, there are a number of relatively professional knowledge facets after searching. Some vague knowledge concept expressions before searching were expressed more clearly and explicitly after searching. In the dimension of knowledge quality, knowledge relevance, analysis and user opinion, the vast majority of users improve after searching. This paper provides a feasible way for researchers to evaluate user knowledge learning in information search from the perspective of multi-angle evaluation of knowledge status before and after searching.
Song Xiaoxuan
,
Liu Chang
. Assessment of User Knowledge Level Before and After Searching[J]. Library and Information Service, 2018
, 62(2)
: 108
-116
.
DOI: 10.13266/j.issn.0252-3116.2018.02.015
[1] RIEH S Y, COLLINS-THOMPSON K, HANSEN P, et al. Towards searching as a learning process[J]. Journal of information science, 2016, 42(1):19-34.
[2] GWIZDKA J, HANSEN P, HAUFF C, et al. Search as learning (SAL) workshop 2016[C]//Proceedings of International ACM SIGIR conference on research and development in information retrieval. New York:ACM, 2016:1249-1250.
[3] RIEH S Y, GWIZDKA J, FREUND L, et al. Searching as learning:novel measures for information interaction research[J]. Proceedings of the American Society for Information Science & Technology, 2014, 51(1):1-4.
[4] MARCHIONINI G. Exploratory search:from finding to understanding[J]. Communications of the ACM, 2006, 49(4):41-46.
[5] 张云秋, 安文秀, 于双成. 探索式搜索中用户认知的实验研究[J]. 情报理论与实践, 2013, 36(6):73-77.
[6] ALLAN J, CROFT B, MOFFAT A, et al. Frontiers, challenges, and opportunities for information retrieval:report from SWIRL 2012 the second strategic workshop on information retrieval in Lorne[J]. Comptes rendus des séances de la société de biologie et de ses filiales, 2012, 161(12):2355-2359.
[7] AGOSTI M, FUHR N, TOMS E, et al. Evaluation methodologies in information retrieval dagstuhl seminar 13441[J]. ACM SIGIR forum, 2014, 48(1):36-41.
[8] WHITE R W, ROTH RA. Exploratory search:beyond the queryresponse paradigm[C]//Proceeding of Synthesis lectures on information concepts retrieval and services. Williston:Morgan & Claypool Publishers, 2009:1-98.
[9] 袁红, 李秋. 搜索任务和搜索能力对用户探索式搜索行为的影响研究[J]. 图书情报工作, 2015, 59(15):94-105.
[10] 张云秋, 安文秀, 冯佳. 探索式信息搜索行为研究[J].图书情报工作, 2012, 56(14):67-72.
[11] EICKHOFF C, TEEVAN J, WHITE R, et al. Lessons from the journey:a query log analysis of within-session learning[C]//Proceedings of ACM conference on Web search and data mining. New York:ACM, 2014:223-232.
[12] COLLINS-THOMPSON K, RIEH S Y, HAYNES C C, et al. Assessing learning outcomes in Web search:a comparison of tasks and query strategies[C]//Proceedings of ACM conference on human information interaction and retrieval. New York:ACM, 2016:163-172.
[13] LAZONDER A W, BIEMANS H J A, WOPEREIS I G J H. Differences between novice and experienced users in searching information on the World Wide Web[J]. Journal of the American society for information science & technology, 2000, 51(6):576-581.
[14] SAITO H, MIWA K. A cognitive study of information seeking processes in the WWW:the effects of searcher's knowledge and experience[C]//Proceedings of the Second international conference on Web information systems engineering (WISE'01) IEEE Computer Society. Washington:IEEE Computer Society, 2001:321-327.
[15] WHITE R W, DUMAIS S T, TEEVAN J. Characterizing the influence of domain expertise on Web search behavior[C]//Proceedings of the second ACM international conference on Web search and data mining. New York:ACM, 2009:132-141.
[16] KOROBILI S, MALLIARI A, ZAPOUNIDOU S. Factors that influence information-seeking behavior:the case of greek graduate students[J]. Journal of academic librarianship, 2011, 37(2):155-165.
[17] LIU C, ZHANG L, SONG X. Are self-assessment of search ability and performance reliable?[C]//Proceedings of the 79th ASIS&T annual meeting:creating knowledge, enhancing lives through information & technology. Maryland:American Society for Information Science Silver Springs, 2016:87.
[18] HERSH W R, ELLIOT D L, HICKAM D H, et al. Towards new measures of information retrieval evaluation[C]//International ACM SIGIR conference on research and development in information retrieval. New York:ACM, 1995:164-170.
[19] NELSON L, HELD C, PIROLLI P, et al. With a little help from my friends:examining the impact of social annotations in sensemaking tasks[C]//Proceedings of the SIGCHI conference on human factors in computing systems. New York:ACM, 2009:1795-1798.
[20] KAMMERER Y, NAIRN R, PIROLLI P, et al. Signpost from the masses:learning effects in an exploratory social tag search browser[C]//Proceedings of the SIGCHI conference on human factors in computing systems. New York:ACM, 2009:625-634.
[21] WILLOUGHBY T, ANDERSON S A, WOOD E, et al. Fast searching for information on the Internet to use in a learning context:the impact of domain knowledge[J]. Computers & education, 2009, 52(3):640-648.
[22] WILSON M J, WILSON M L. A comparison of techniques for measuring sensemaking and learning within participant-generated summaries[J]. Journal of the American Society for Information Science & Technology, 2013, 64(2):291-306.
[23] BLOOM B S, ENGELHART M D, FURST E J, et al. Taxonomy of educational objectives:the classification of educational goals. Handbook I:Cognitive domain[M]. London:Longmans,1956:58.
[24] LEE J. Does higher education foster critical and creative learners? An exploration of two universities in South Korea and the USA[J]. Higher education research & development, 2015, 34(1):1-16.
[25] 刘畅, 赵瑜, 杨帆. 信息检索用户实验设计中时间限制和任务次序的影响研究[J]. 图书情报工作, 2015, 59(1):99-105.
[26] LIU J, BELKIN N J, ZHANG X, et al. Examining users' knowledge change in the task completion process[J]. Information processing & management, 2013, 49(5):1058-1074.
