A Study on Big Data-based GraphX Model for Social Network Service
0
조회
0
다운로드
2
인용
0
참고문헌
키워드
초록
Nowaday, towards adopting big data processing system has increased and it is commonly seen in every aspect of life. For this, the problem of finding connected components in undirected graphs has been well studied, and it is an essential pre-processing step to many graph computations, and a fundamental task in graph analytics applications. Recently, it has been a main area of interest in the large graph processing. However, much of the research has focused on solving the problem using High Performance Computers. In large distributed systems, the MapReduce framework dominates the processing of big data, and has been used for finding connected components in big graphs although iterative processing is not directly supported in MapReduce. Current big data processing systems have developed into supporting iterative processing and providing additional features other than MapReduce. This research investigates how to enhance the performance of finding connected components algorithm for large graph in distributed processing system. It uses the approach to considering the graph degree property in choosing the component identifier, reviewing how this can affect the efficiency of the algorithm. In the design of our proposed algorithm features provided by current new processing systems such as moving the computation more toward the data partition in Spark framework model are integrated.
Abstract
Nowaday, towards adopting big data processing system has increased and it is commonly seen in every aspect of life. For this, the problem of finding connected components in undirected graphs has been well studied, and it is an essential pre-processing step to many graph computations, and a fundamental task in graph analytics applications. Recently, it has been a main area of interest in the large graph processing. However, much of the research has focused on solving the problem using High Performance Computers. In large distributed systems, the MapReduce framework dominates the processing of big data, and has been used for finding connected components in big graphs although iterative processing is not directly supported in MapReduce. Current big data processing systems have developed into supporting iterative processing and providing additional features other than MapReduce. This research investigates how to enhance the performance of finding connected components algorithm for large graph in distributed processing system. It uses the approach to considering the graph degree property in choosing the component identifier, reviewing how this can affect the efficiency of the algorithm. In the design of our proposed algorithm features provided by current new processing systems such as moving the computation more toward the data partition in Spark framework model are integrated.
참고문헌 (15)
- 황성태, "A Study on Big Data Platform Architecture-based Conceptual Measurement Model Using Comparative Analysis for Social Commerce", 한국지식정보기술학회 논문지, 15(5), pp.623~630 (2020), 한국지식정보기술학회[DOI]
- 김정식, "A Study on Adaptive Smart Platform for Intelligent Software in Big Data Environment", 한국지식정보기술학회 논문지, 15(3), pp.347~355 (2020), 한국지식정보기술학회[DOI]
- 장효경, "빅데이터 환경에서 개인화 서비스를 위한 상황인식 기법", 한국지식정보기술학회 논문지, 15(5), pp.631~638 (2020), 한국지식정보기술학회[DOI]
- T. Akidau, "The dataflow model: A practical approach to balancing correctness, latency, and cost in massive-scale, unbounded, out-of-order data processing", Proceeding VLDB Endow, 8(12), pp.1792~1803 (2015)
- P. Alvaro, "Consistency analysis in bloom:A calm and collected approach", Proceedings 5th Biennial Conference on Innovative Data Systems Research, pp.249~260 (2011)
- B. Schilit, "Context -aware computing applications", First Work shop on Mobile Computing Systems and Applications, pp.85~90 (1994)
- D. Salber, "The context toolkit: aiding the development of context Aware applications", the Workshop on Software Engineering for Wearable and Pervasive Computing (2000)
- N. Pandeeswari, "Anomaly detection system in cloud environment using fuzzy clustering based ANN", Mob. Netw. Appl, 21(3), pp.494~505 (2016)
- R. J. Hyndman, "Large-scale unusual time series detection", 2015 IEEE International Conference on Data Mining Workshop (ICDMW), pp.1616~1619 (2015), IEEE
- B. Agrawal, "R2Time: A framework to analyse open TSDB time-series data in HBase", Cloud Computing Technology and Science (CloudCom), 2014 IEEE 6th International Conference, pp.970~975 (2014)
- C. Wang, "Online detection of utility cloud anomalies using metric distributions", Network Operations and Management Symposium (NOMS), 2010 IEEE, pp.96~103 (2010)
- M. Jahrer, "Ensemble of collaborative filtering and feature engineered models for click through rate prediction", KDDCup Workshop (2012)
- P. Gaikwad, "Anomaly detection for scientific workflow applications on networked clouds", High Performance Computing &Simulation (HPCS), 2016 International Conference on, pp.645~652 (2016), IEEE
- U. Kang, "Mining large graphs: Algorithms, inference, and discoveries", 2011 IEEE 27th International Conference on Data Engineering, pp.243~254 (2011)
- T. Rabl, "Advancing Big Data Benchmarks, Vol. 8585" (2014), Springer International Publishing