병렬 딥러닝에 기반한 웹 접근 예측 최적화
Web access prediction based on parallel deep learning
0
조회
0
다운로드
0
인용
0
참고문헌
키워드
초록
Due to the exponential growth of access information on the web, the need for predicting web users’ next access has increased. Various models such as markov models, deep neural networks, support vector machines, and fuzzy inference models were proposed to handle web access prediction. For deep learning based on neural network models, training time on large-scale web usage data is very huge. To address this problem, deep neural network models are trained on cluster of computers in parallel. In this paper, we investigated impact of several important spark parameters related to data partitions, shuffling, compression, and locality (basic spark parameters) for training Multi-Layer Perceptron model on Spark standalone cluster. Then based on the investigation, we tuned basic spark parameters for training Multi-Layer Perceptron model and used it for tuning Spark when training Multi-Layer Perceptron model for web access prediction. Through experiments, we showed the accuracy of web access prediction based on our proposed web access prediction model. In addition, we also showed performance improvement in training time based on our spark basic parameters tuning for training Multi-Layer Perceptron model over default spark parameters configuration.
Abstract
Due to the exponential growth of access information on the web, the need for predicting web users’ next access has increased. Various models such as markov models, deep neural networks, support vector machines, and fuzzy inference models were proposed to handle web access prediction. For deep learning based on neural network models, training time on large-scale web usage data is very huge. To address this problem, deep neural network models are trained on cluster of computers in parallel. In this paper, we investigated impact of several important spark parameters related to data partitions, shuffling, compression, and locality (basic spark parameters) for training Multi-Layer Perceptron model on Spark standalone cluster. Then based on the investigation, we tuned basic spark parameters for training Multi-Layer Perceptron model and used it for tuning Spark when training Multi-Layer Perceptron model for web access prediction. Through experiments, we showed the accuracy of web access prediction based on our proposed web access prediction model. In addition, we also showed performance improvement in training time based on our spark basic parameters tuning for training Multi-Layer Perceptron model over default spark parameters configuration.
참고문헌 (16)
- Mamoun A. Awad, "Prediction of User’s web-browsing behavior : Application of Markov Model", IEEE Transactions on Systems, Man, And Cybernetics – Part B: Cybernetic, 42(4), pp.1131~1142 (2012)
- "Web Mining and Knowledge Discovery of Usage Patterns" (2000)[URL]
- Giovanna Castellano, "Advanced Techniques in Web Intelligence-2", pp.75~104 (2013)
- M. Zaharia, "Resilient distributed datasets:a fault-tolerant abstraction for in-memory cluster computing", 9th USENIX Symposium on Networked Systems Design and Implementation (NSDI), pp.15~28 (2012)
- Anastasios Gounaris, "A Methodlogy for Spark Parameter Tuning", Big Data Research, 11, pp.22~32 (2018)
- P. Petridis, "Spark parameter tuning via trial-and-error", Advances in Big Data – Proceedings of the 2nd INNS Conference on Big Data, pp.226~237 (2016)
- "Spark guidelines documentatin for tunning"[URL]
- R. Tous, "Spark deployment and performance evaluation on the marenostrum supercomputer", IEEE International Conference on Big Data (Big Data), pp.299~306 (2015)
- "Alpine Data tuning tip"[URL]
- A.J. Awan, "How data volume affects spark data data analysitcs on a scale-up server", ~ (2015)
- "Spark parameters configuration"[URL]
- Om Prakash Mandal, "Web Access Prediction Model using Clustering and Artificial Neural Network", IJERT, 3(9) (2014)
- Pruthvi, "Web-Users’ Browsing behavior Prediction by Implementing Neural Network in MapReduce", IJAFRC, 1(5) (2014)
- Vidushi, Yashpal Singh, "SOM Improved Neural Network Approach for Next Page Prediction", International Journal of Computer Science and Mobile Computing, 4(5), pp.175~181 (2015)
- "NASA : web access log dataset"[URL]
- "ClarkNet: web access log dataset"[URL]