RHadoop 플랫폼에서 로지스틱 회귀 알고리즘 성능 비교
Performance Comparison of Logistic Regression Algorithms on RHadoop
0
조회
0
다운로드
2
인용
0
참고문헌
키워드
초록
Machine learning has found widespread implementations and applications in many different domains in our life. Logistic regression is a type of classification in machine leaning, and is used widely in many fields, including medicine, economics, marketing and social sciences. In this paper, we present the MapReduce implementation of three existing algorithms, this is, Gradient Descent algorithm, Cost Minimization algorithm and Newton-Raphson algorithm, for logistic regression on RHadoop that integrates R and Hadoop environment applicable to large scale data. We compare the performance of these algorithms for estimation of logistic regression coefficients with real and simulated data sets. We also compare the performance of our RHadoop and RHIPE platforms. The performance experiments showed that our Newton-Raphson algorithm when compared to Gradient Descent and Cost Minimization algorithms appeared to be better to all data tested, also showed that our RHadoop was better than RHIPE in real data, and was opposite in simulated data.
Abstract
Machine learning has found widespread implementations and applications in many different domains in our life. Logistic regression is a type of classification in machine leaning, and is used widely in many fields, including medicine, economics, marketing and social sciences. In this paper, we present the MapReduce implementation of three existing algorithms, this is, Gradient Descent algorithm, Cost Minimization algorithm and Newton-Raphson algorithm, for logistic regression on RHadoop that integrates R and Hadoop environment applicable to large scale data. We compare the performance of these algorithms for estimation of logistic regression coefficients with real and simulated data sets. We also compare the performance of our RHadoop and RHIPE platforms. The performance experiments showed that our Newton-Raphson algorithm when compared to Gradient Descent and Cost Minimization algorithms appeared to be better to all data tested, also showed that our RHadoop was better than RHIPE in real data, and was opposite in simulated data.
참고문헌 (16)
- J. M. Hilbe, "Logistic Regression Models", pp.2009, Chapman & Hall
- B. Oancea, "Integration R and Hadoop for Big data analysis", Romanian Statistical Review, (2), pp.83~94 (2014)
- T. White, "Hadoop: The Definitive Guide" (2012), O’Reilly Media, Inc
- V. Prajapati, "Big data analytics with R and Hadoop" (2013), Packt Publishing Ltd
- S. Guha, "Computing environment for the statistical analysis of large and complex data", PhD, Department of Statistics, pp.2010
- S. Guha, "Large complex data: divide and recombine (D&R) with RHIPE", Stat, 191, pp.53~67 (2012)
- 고영준, "Rhipe를 활용한 빅데이터 처리 및 분석", 한국데이터정보과학회지, 24(5), pp.975~987 (2013), 한국데이터정보과학회[DOI]
- 정병호, "빅데이트 처리 및 분석을 위한 Rhipe 플랫폼", 응용통계연구, 27(7), pp.1171~1185 (2014), 한국통계학회
- D. Harish, "Big data analysis using Rhadoop", IJIRAE, 4(2), pp.180~185 (2015)
- 신지은, "RHadoop을 이용한 빅데이터 분산처리 시스템", 한국데이터정보과학회지, 26(5), pp.1155~1166 (2015), 한국데이터정보과학회[DOI]
- R. Uskenbаyevа, "Integrаting of dаtа using the Hаdoop аnd R", Procedia Computer Science, 56, pp.145~149 (2015)
- J. Wu, "Foundations of Predictive Analytics", pp.2012, Chapman and Hall
- E. Sammer, "Hadoop Operations" (2012), O’Reilly Media, Inc
- ""Airline on-time performance," ASA Section on: Statistical Computing Statistical Graphics"[URL]
- C. Wang, "A survey of statistical methods and computing for Big Data" (2015), Cornell University Library
- M. Rashid, "Inference on logistic regression models", Doctor, pp.2008