Personalized Product Recommendation System Based on Spark
1
조회
0
다운로드
1
인용
0
참고문헌
키워드
초록
With the rapid development of the Internet, the way consumers shop has undergone tremendous changes. Compared with traditional physical store shopping methods, online shopping methods are more convenient and faster, so e-commerce platforms are becoming more and more popular. With the increase in the number of users and products on the e-commerce platform, the information faced by users is becoming more and more diverse. To quickly find the products that users are interested in, an e-commerce recommendation system was born. The e-commerce recommendation system recommends corresponding products according to the user's preferences, which can save the user's time in selecting products and increase the user's stickiness to the e-commerce platform. Recommendation systems mainly include recommendation systems based on Hadoop MapReduce computing, recommendation systems based on Spark computing, etc. The recommendation systems based on Spark computing have the advantages of high real-time performance and computational efficiency. This paper uses big data technologies such as Hadoop, Spark, and Kafka, combined with algorithms such as collaborative filtering and K-means, to implement a personalized product recommendation system with a real-time recommendation module and an offline recommendation module. Through experimental comparison, the performance of the offline recommendation algorithm in this system is better than the traditional user-based collaborative filtering algorithm. At the same time, the experiment also proves that the Spark computing framework has higher computational efficiency than the Hadoop MapReduce computing framework.
Abstract
With the rapid development of the Internet, the way consumers shop has undergone tremendous changes. Compared with traditional physical store shopping methods, online shopping methods are more convenient and faster, so e-commerce platforms are becoming more and more popular. With the increase in the number of users and products on the e-commerce platform, the information faced by users is becoming more and more diverse. To quickly find the products that users are interested in, an e-commerce recommendation system was born. The e-commerce recommendation system recommends corresponding products according to the user's preferences, which can save the user's time in selecting products and increase the user's stickiness to the e-commerce platform. Recommendation systems mainly include recommendation systems based on Hadoop MapReduce computing, recommendation systems based on Spark computing, etc. The recommendation systems based on Spark computing have the advantages of high real-time performance and computational efficiency. This paper uses big data technologies such as Hadoop, Spark, and Kafka, combined with algorithms such as collaborative filtering and K-means, to implement a personalized product recommendation system with a real-time recommendation module and an offline recommendation module. Through experimental comparison, the performance of the offline recommendation algorithm in this system is better than the traditional user-based collaborative filtering algorithm. At the same time, the experiment also proves that the Spark computing framework has higher computational efficiency than the Hadoop MapReduce computing framework.
참고문헌 (20)
- M, Liu, H. Liu, and L. Li, Research on hadoop massive power quality data based on computer cloud platform, Journal of Physics : Conference Series, Vol, "Research on hadoop massive power quality data based on computer cloud platform", Journal of Physics: Conference Series, 1915(2), pp.22~40 (2021)
- D. L. Chiang, "Modeling and analysis of hadoop mapreduce systems for big data using petri nets", pp.80~104 (2021), Applied Artificial Intelligence
- 정원일, "스파크 프레임워크에서 연속집계 질의처리를 위한 자원공유기법", 한국지식정보기술학회 논문지, 15(5), pp.671~681 (2020), 한국지식정보기술학회[DOI]
- X. W. Zhao, "We know what you want to buy: a demographic-based system for product recommendation on microblogs", Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining, pp.1935~1944 (2014)
- T. Badriyah, "A hybrid recommendation system for E-commerce based on product description and user profile", 2017 Seventh International Conference on Innovative Computing Technology, pp.95~100 (2017), IEEE
- B. Y. Chae, "Development of product recommendation system Using CBF based on big data and application of chatbot service", Korea Information Processing Society review, pp.50~58 (2021)
- J. Shen, "Collaborative filtering-based recommendation system for big data", International Journal of Computational Science and Engineering, pp.219~225 (2020)
- Z. D. Zhao, "User-based collaborative-filtering recommendation algorithms on hadoop", 2010 third international conference on knowledge discovery and data mining, pp.478~481 (2010)
- J. Jiang, "Scaling-up item-based collaborative filtering recommendation algorithm based on hadoop", 2011 IEEE World Congress on Services, pp.490~497 (2011)
- C. Charu, "Model-based collaborative filtering", pp.71~138 (2016), Recommender systems
- S. Kavitha, "Towards a hybrid recommendation system on Apache Spark", 2020 IEEE India Council International Subsections Conference (INDISCON), pp.297~302 (2020), IEEE
- J. Wei, "Collaborative filtering and deep learning based recommendation system for cold start items", pp.29~39 (2017), Expert Systems with Applications
- M. N. Vora, "Hadoop-HBase for large-scale data", Proceedings of 2011 International Conference on Computer Science and Network Technology, 1, pp.601~605 (2011)
- M. Andreea, "Big data analytics: analysis of features and performance of big data ingestion tools" (2018), Informatica Economica
- J. H. Hong, "Design of video surveillance system using k-means clustering,", Journal of The Korea Internet of Things Society, pp.1~5 (2017)
- J. Hu, "Hybrid recommendation algorithm based on latent factor model and PersonalRank", Journal of Internet Technology, pp.919~926 (2018)
- F. Liu, "Design of personalized recommendation algorithms based on big data framework", Journal of Physics:Conference Series, 2138(1), pp.012025~ (2021)
- S. Gosh, "Recommendation system for e-commerce using alternating least squares(als) on apache spark", International Conference on Intelligent Computing & Optimization, pp.880~893 (2020)
- 김철진, "상품 추천 기법의 성능평가 분석", 한국지식정보기술학회 논문지, 14(5), pp.515~525 (2019), 한국지식정보기술학회[DOI]
- 강부식, "Word2Vec을 이용한 사용자기반 협업필터링의 예측 정확도 개선", 한국지식정보기술학회 논문지, 13(1), pp.169~176 (2018), 한국지식정보기술학회[DOI]