Username   Password       Forgot your password?  Forgot your username? 


Collaborative Filtering Recommendation Algorithm based on Spark

Volume 15, Number 3, March 2019, pp. 930-938
DOI: 10.23940/ijpe.19.03.p22.930938

Jinhong Taoa, Jianhou Ganb, and Bin Wena

aSchool of Information Science and Technology, Yunnan Normal University, Kunming, 650500, China
bKey Laboratory of Educational Informatization for Nationalities of Ministry of Education, Yunnan Normal University, Kunming, 650500, China

(Submitted on November 16, 2018; Revised on December 15, 2018; Accepted on January 11, 2019)


With the advent of the era of big data, the problem of information overload has become particularly serious. The recommendation system can provide personalized recommendation services for users by analyzing users’ basic information and users’ behavior information. How to push information accurately and efficiently has become an urgent issue in the era of big data. Based on the Alternating Least Squares (ALS) collaborative filtering recommendation algorithm, this paper reduces the loss of the invisible factor item attribute information by merging the similarity of the item on the loss function. At the same time, the cold start strategy is introduced into the model to complete the recommendation. The algorithm is implemented on the Spark distributed platform and single node by using the Movie Lens dataset published by the GroupLens Lab. The experiment results show that the proposed recommendation algorithm can preferably alleviate the data sparsity problem compared with the traditional recommendation algorithm. Moreover, the algorithm improves the accuracy of recommendation and the efficiency of calculation.


References: 24

        1. F. Ricci, L. Rokach, and B. Shapira, “Recommender Systems Handbook,” 2nd Edition, Springer-Verlag New York Inc., New York, USA, November 2015
        2. D. S. Li, C. Chen, Q. Lv, and H. S. Gu, “An Adaptive Learning Rate Method for Matrix Approximation-based Collaborative Filtering,” in Proceedings of 2018 World Wide Web Conferences, pp. 741-751, Lyon, France, July 2018
        3. N. E. I. Karabadji, S. Beldjoudi, H. Serid, S. Aridhi, and W. Dhifli, “Improving Memory-based User Collaborative Filtering with Evolutionary Multi-Objective Optimization,” Expert Systems with Applications, Vol. 98, pp. 153-165, May 2018
        4. B. Shams and S. Haratizadeh, “Item-based Collaborative Ranking,” Knowledge-based Systems, Vol. 152, pp. 172-185, July 2018
        5. T. M. Huynh, H. H. Huynh, V. T. Tran, and H. X. Huynh, “Collaborative Filtering Recommender System base on The Interaction Multi-Criteria Decision with Ordered Weighted Averaging Operator,” in Proceedings of the 2nd International Conference on Machine Learning and Soft Computing, pp. 45-49, New York, USA, February 2018
        6. E. Karydi and K. G. Margaritis, “Parallel and Distributed Collaborative Filtering: A Survey,” ACM Computing Surveys Surveys Homepage Archive (CSUR), Vol. 49, No. 2, pp. 1-46, New York, USA, November 2016
        7. Y. Wang and L. He, “Research and Optimization of Data Sparsity in Collaborative Filtering Algorithms,” Recent Developments in Intelligent Computing, Communication and Devices, Vol. 752, pp. 87-92, Singapore, August 2018
        8. J. Xu, Y. S. Zhong, W. Q. Zhu, and F. F. Sun, “Trust-based Context-Aware Mobile Social Network Service Recommendation,” Wuhan University Journal of Natural Sciences, Vol. 22, No. 2, pp. 149-156, Wuhan, China, April 2017
        9. Y. C. Jing, W. Jiang, G. Y. Su, Z. S. Zhou, and Y. F. Wang, “A Learning Automata-based Singular Value Decomposition and its Application in Recommendation System,” Springer International Publishing, pp. 26-32, Taiyuan, China, August 2014
        10. Z. J. Wang, N. N. Yu, and J. X. Wang, “User Attributes Clustering-based Collaborative Filtering Recommendation Algorithm and its Parallelization on Spark,” in Proceedings of Asian Simulation Conference, pp. 442-451, Beijing, China, September 2016
        11. L. Zhang, X. Song, and Y. J. Wu, “Theory, Methodology, Tools and Applications for Modeling and Simulation of Complex Systems,” in Proceedings of 16th Asia Simulation Conference and SCS Autumn Simulation Multi-Conference, Beijing, China, October 2016
        12. C. Guan and K. K. F. Yuen, “Towards a Hybrid Approach of Primitive Cognitive Network Process and Agglomerative Hierarchical Clustering for Music Recommendation,” in Proceedings of 11th International Conference on Heterogeneous Networking for Quality, Reliability, Security and Robustness (QSHINE), pp. 206-209, Taipei, Taiwan, November 2015
        13. Z. D. Zhao and M. S. Shang, “User-based Collaborative Filtering Recommendation Algorithms on Hadoop,” in Proceedings of International Conference on Knowledge Discovery & Data Mining, pp. 478-481, Phuket, Thailand, January 2010
        14. L. Fan, H. Li, and C. F. Li, “The Improvement and Implementation of Distributed Item-based Collaborative Filtering Algorithm on Hadoop,” in Proceedings of 34th Chinese Control Conference (CCC), pp. 9078-9083, Hangzhou, China, July 2015
        15. K. H. Lin, J. J. Wang, and M. H. Wang, “A Hybrid Recommendation Algorithm based on Hadoop,” in Proceedings of 9th International Conference on Computer Science and Education, pp. 540-543, Vancouver, BC, Canada, August 2014
        16. B. Kupisz and O. Unold, “Collaborative Filtering Recommendation Algorithm based on Hadoop and Spark,” in Proceedings of 2015 IEEE International Conference on Industrial Technology (ICIT), pp. 1510-1514, Seville, Spain, June 2015
        17. S. Salloum, R. Dautov, X. J. Chen, P. X. G. Peng, and J. Z. X. Huang, “Big Data Analytics on Apache Spark,” International Journal of Data Science and Analytics, Vol. 1, No. 3-4, pp. 145-164, November 2016
        18. Y. Samadi, M. Zbakh, and C. Tadonki, “Performance Comparison Between Hadoop and Spark Frameworks using HiBench Benchmarks,” Concurrency and Computation Practice & Experience, pp. 1-13, Wiley, USA, November 2017
        19. R. K. Mishra, “Spark Architecture and The Resilient Distributed Dataset,” PySpark Recipes, pp. 85-114, Apress, Berkeley, CA, December 2017
        20. D. G. García, S. G. Ramírez, and S. García, “A Comparison on Scalability for Batch Big Data Processing on Apache Spark and Apache Flink,” Big Data Anal, Vol. 2, No. 1, pp. 1, March 2017
        21. I. S. Wahyudi, A. Affandi, and M. Hariadi, “Recommender Engine using Cosine Similarity based on Alternating Least Square-Weight Regularization,” in Proceedings of 15th International Conference on Quality in Research (QiR), International Symposium on Electrical and Computer Engineering, pp. 256-261, Nusa Dua, Indonesia, July 2017
        22. C. Verma and R. Pandey, “Big Data Representation for Grade Analysis Through Hadoop Framework,” in Proceedings of 6th International Conference - Cloud System and Big Data Engineering (Confluence), pp. 312-315, Noida, India, January 2016
        23. R. Katarya and O. P. Verma, “Recommender System with Grey Wolf Optimizer and FCM,” Neural Computing & Applications, Vol. 30, No. 5, pp. 1679-1687, September 2018
        24. C. Selvi and E. Sivasankar, “A Novel Singularity based Improved Tanimoto Similarity Measure for Effective Recommendation using Collaborative Filtering,” in Proceedings of 2018 8th International Conference on Cloud Computing, Data Science & Engineering (Confluence), pp. 256-262, Noida, India, January 2018


        Please note : You will need Adobe Acrobat viewer to view the full articles.Get Free Adobe Reader

        This site uses encryption for transmitting your passwords.