Library and Information Service >
Research on Collaborative Filtering Book Recommendation Based on Hadoop and Mahout
Received date: 2013-08-05
Revised date: 2013-08-21
Online published: 2013-09-20
Firstly, this paper builds a book recommendation engine system based on the Hadoop open source distributed computing framework and mahout collaborative filtering recommendation engine technology. Then it takes advantage of the cloud model and Pearson coefficient to improve the traditional collaborative filtering recommendation algorithm, and resolves the problems of poor system performance and recommendation results inaccurate of traditional stand-alone recommendation algorithm in high-dimensional sparse matrix operations. Thirdly, it experiments and evaluates the overall performance of the distributed recommendation platform and the improved collaborative filtering algorithm. It finds that: (1) when the virtual machine nodes are increasing, the computation time of collaborative filtering recommendation engine is declining in the experimental tests, which shows that the overall performance of the system has been improved. (2) it improves the mahout original collaborative filtering recommendation engine with the Pearson coefficient and evaluates the recommended effect with MAE indices of the original collaborative filtering recommendation algorithm, which finds the recommendation accuracy rate increases 13.1% and the subjectivity differences of user ratings have great impact on the recommendation accuracy.
Key words： book recommendation; Hadoop; Mahout; recommendation engine; collaborative filtering
Feng Guohe , Huang Jiaxing . Research on Collaborative Filtering Book Recommendation Based on Hadoop and Mahout[J]. Library and Information Service, 2013 , 57(18) : 116 -121 . DOI: 10.7536/j.issn.0252-3116.2013.18.020
[1] Goldberg D, Nichols D, Oki B M, et al. Using collaborative filtering to weave aninformation tapestry.[J]. Communications of the ACM,1992,35(12): 61-70.
[2] Breese I S,Heckeman D, Kadie C.Empirical analysis of predictive algorithms for collaborative filtering[C]//Proceedings of the 14th Conference on Uncertainty in Artificial Intelligence.San Francisco:ACM Press,1998:43-52.
[3] 郁雪.基于协同过滤技术的推荐方法研究[D].天津:天津大学,2009.
[4] 奉国和,梁晓婷.协同过滤推荐研究综述[J].图书情报工作,2011,55(16):127-130.
[5] Sarwar B, Karypis G, Konstan J,et al.Item-based collaborative filtering recommendation algorithms[C]//Proceedings of the 10th International World Wide Web Conference. New York:ACM Press,2001:285-295.
[6] 利用Cloudera实现Hadoop[EB/OL].[2012-09-24].http://wiki.ubuntu.org.cn/利用Cloudera实现Hadoop.
[7] Apache Software Foundation[EB/OL].[2013-08-04].http://www.apache.org/.
[8] Secure Shell[EB/OL].[2012-09-19]. http://zh.wikipedia.org/wiki/Secure_Shell.
[9] JDK download[CP/OL].[2012-12-04].http://www.oracle.com/technetwork/java/javase/downloads/jdk-6u27-download-440405.html
[10] Ziegler C N,McNee S M,Konstan J A, et al.Improving recommendation lists through topic diversification[EB/OL].[2013-08-20]. http://dl.acm.org/citation.cfm?id=1060754.
[11] Book-Crossing Dataset[EB/OL].[2012-12-09].http://www.informatik.uni-freiburg.de/~cziegler/BX/.
[12] SequenceFile[EB/OL].[2012-12-23].http://wiki.apache.org/hadoop/SequenceFile.
[13] Yu Chuan,Xu Jieping,Du Xiaoyong.Recommendation algorithm combining the user-basedclassified regression and the item-based filtering[EB/OL].[2013-08-20]. http://dl.acm.org/citation.cfm?id=1151463.
[14] Herlocker J L,Konstan J A,Terveen L G,et al.Evaluating collaborative filtering recommender system[J].ACM Transactions on Information System,2004,22(1):5-53.
[15] 刘建国,周涛,郭强,等.个性化推荐系统评价方法综述[J].复杂系统与复杂性科学,2009,6(3):1-8.
/
| 〈 |
|
〉 |
