빅데이터 통합 방법에 관한 연구
Research on Big Data Integration Method
0
조회
0
다운로드
0
인용
0
참고문헌
키워드
초록
In this paper we propose the approach for big data integration so as to analyze, visualize and predict the future of the trend of the market, and that is to get the integration data model using the R language which is the future of the statistics and the Hadoop which is a parallel processing for the data. As four approaching methods using R and Hadoop, ff package in R, R and Streaming as Hadoop utility, and Rhipe and RHadoop as R and Hadoop interface packages are used, and the strength and weakness of four methods are described and analyzed, so Rhipe and RHadoop are proposed as a complete set of data integration model. The integration of R, which is popular for processing statistical algorithm and Hadoop contains Distributed File System and resource management platform and can implement the MapReduce programming model gives us a new environment where in R code can be written and deployed in Hadoop without any data movement. This model allows us to predictive analysis with high performance and deep understand over the big data.
Abstract
In this paper we propose the approach for big data integration so as to analyze, visualize and predict the future of the trend of the market, and that is to get the integration data model using the R language which is the future of the statistics and the Hadoop which is a parallel processing for the data. As four approaching methods using R and Hadoop, ff package in R, R and Streaming as Hadoop utility, and Rhipe and RHadoop as R and Hadoop interface packages are used, and the strength and weakness of four methods are described and analyzed, so Rhipe and RHadoop are proposed as a complete set of data integration model. The integration of R, which is popular for processing statistical algorithm and Hadoop contains Distributed File System and resource management platform and can implement the MapReduce programming model gives us a new environment where in R code can be written and deployed in Hadoop without any data movement. This model allows us to predictive analysis with high performance and deep understand over the big data.
참고문헌 (18)
- 조영임, "빅데이터의 이해와 주요 이슈들", 한국지역정보화학회지, 16(3), pp.31~52 (2013), 한국지역정보화학회
- Piyush Gupta, "Sentiment Analysis on Hadoop with Hadoop Streaming", International Journal of Computer Applications, 121(11) (2015)
- 고영준, "Rhipe를 활용한 빅데이터 처리 및 분석", 한국데이터정보과학회지, 24(5), pp.975~987 (2013), 한국데이터정보과학회[DOI]
- Anju Gahlawat, "Big Data Analysis using R and Hadoop", IJCEM International Journal of Computational Engineering & Management, 17(5) (2014)
- Oracle: Big Data for the Enterprise, "Jean-Pierre Dijcks", Oracle, 3~4 (2013)
- "R Tools Evaluation" (2015)[URL]
- "Understanding Hadoop clusters and the network"[URL]
- D. Adler, "The ff package: Handling Large Data Sets in R with Memory Mapped Pages of Binary Flat Files", Institute for Statistics and Econometrics (2007)
- "Hadoop Streaming" (2007)[URL]
- "The Revolution Anаlytics Perspective on Big Dаtа"[URL]
- "Big Data: the new ‘The Future’" (2015)[URL]
- The next frontier for innovation, competition and productivity, "Manyika J", McKinsey &Company, ~ (2011)
- Zikopoulos P, "Understanding big data: Analytics for enterprise class Hadoop and streaming data" (2012), McGraw-Hill
- Integrating and governing big data, "David Corrigan", IBM, ~ (2013)
- "Oracle: Hadoop Based Processing" (2012)[URL]
- "Implementation of MapReduce in R" (2016)[URL]
- Choonghyun You, "Technology Trends in Big Data Analytics and Introduction to R", NexR, Data Science Team, KRNet (2012)
- Kevin, "Large Data Analysis Using Rhipe", RHadoop, 2013