Enhanced Regular Expression as a DGL for Generation of Synthetic Big Data
1
조회
0
다운로드
0
인용
0
참고문헌
키워드
초록
Synthetic data generation is generally used in performance evaluation and function tests in data-intensiveapplications, as well as in various areas of data analytics, such as privacy-preserving data publishing (PPDP)and statistical disclosure limit/control. A significant amount of research has been conducted on tools andlanguages for data generation. However, existing tools and languages have been developed for specificpurposes and are unsuitable for other domains. In this article, we propose a regular expression-based datageneration language (DGL) for flexible big data generation. To achieve a general-purpose and powerful DGL,we enhanced the standard regular expressions to support the data domain, type/format inference, sequence andrandom generation, probability distributions, and resource reference. To efficiently implement the proposedlanguage, we propose caching techniques for both the intermediate and database queries. We evaluated theproposed improvement experimentally.
Abstract
Synthetic data generation is generally used in performance evaluation and function tests in data-intensiveapplications, as well as in various areas of data analytics, such as privacy-preserving data publishing (PPDP)and statistical disclosure limit/control. A significant amount of research has been conducted on tools andlanguages for data generation. However, existing tools and languages have been developed for specificpurposes and are unsuitable for other domains. In this article, we propose a regular expression-based datageneration language (DGL) for flexible big data generation. To achieve a general-purpose and powerful DGL,we enhanced the standard regular expressions to support the data domain, type/format inference, sequence andrandom generation, probability distributions, and resource reference. To efficiently implement the proposedlanguage, we propose caching techniques for both the intermediate and database queries. We evaluated theproposed improvement experimentally.
참고문헌 (39)
- A. Adir, "Hardware and Software: Verification and Testing", pp.219~233 (2012), Springer
- T. S. Buda, "VFDS: an application to generate fast sample databases", Proceedings of the 23rd ACM International Conference on Information and Knowledge Management, pp.2048~2050 (2014)
- T. Rabl, "Performance Evaluation, Measurement and Characterization of Complex Systems", pp.41~56 (2011), Springer
- T. Rabl, "Just can't get enough: synthesizing big data", Proceedings of the 2015 ACM SIGMOD International Conference on Management of Data, pp.1457~1462 (2015)
- K. Taneja, "MODA: automated test generation for database applications via mock objects", Proceedings of the IEEE, 2010, 289~292
- H. Wu, "Generating realistic synthetic population datasets", ACM Transactions on Knowledge Discovery from Data (TKDD), 12(4), pp.1~22 (2018)
- K. Mason, "An ʺon the flyʺ framework for efficiently generating synthetic big data sets", Proceedings of 2019 IEEE International Conference on Big Data (Big Data), pp.3379~3387 (2019)
- B. C. Fung, "Privacy-preserving data publishing: a survey of recent developments", ACM Computing Surveys, 42(4), pp.1~53 (2010)
- "The future of statistical disclosure control" (2018)[URL]
- A. Dries, "Declarative data generation with problog", Proceedings of the 6th International Symposium on Information and Communication Technology (SoICT), pp.17~24 (2015)
- D. C. Ince, "The automatic generation of test data", The Computer Journal, 30(1), pp.63~69 (1987)
- J. E. Hoag, "A parallel general-purpose synthetic data generator", ACM SIGMOD Record, 36(1), pp.19~24 (2007)
- L. Burnett, "The "GeneTrustee": a universal identification system that ensures privacy and confidentiality for human genetic databases", Journal of Law and Medicine, 10(4), pp.506~513 (2003)
- H. Surendra, "A review of synthetic data generation methods for privacy preserving data publishing", International Journal of Scientific & Technology Research, 6(3), pp.95~101 (2017)
- "Comparative evaluation of synthetic data generation methods" (2017)[URL]
- J. Fan, "Relational data synthesis using generative adversarial networks: a design space exploration", Proceedings of the VLDB Endowment, 13(11), pp.1962~1975 (2020)
- Ruchika Malhotra, "An Adequacy Based Test Data Generation Technique Using Genetic Algorithms", JIPS(Journal of Information Processing Systems), 7(2), pp.363~384 (2011), 한국정보처리학회
- Sangeeta Sabharwal, "Test Set Generation for Pairwise Testing Using Genetic Algorithms", JIPS(Journal of Information Processing Systems), 13(5), pp.1089~1102 (2017), 한국정보처리학회[DOI]
- Paul Fisher, "Generation of Finite Inductive, Pseudo Random, Binary Sequences", JIPS(Journal of Information Processing Systems), 13(6), pp.1554~1574 (2017), 한국정보처리학회[DOI]
- 곽정훈, "Path Generation Method of UAV Autopilots using Max-Min Algorithm", JIPS(Journal of Information Processing Systems), 14(6), pp.1457~1463 (2018), 한국정보처리학회[DOI]
- J. Gray, "Quickly generating billion-record synthetic databases", Proceedings of the 1994 ACM SIGMOD International Conference on Management of Data, pp.243~252 (1994)
- J. M. Stephens, "MUDD: a multi-dimensional data generator", ACM SIGSOFT Software Engineering Notes, 29(1), pp.104~109 (2004)
- N. Bruno, "Flexible database generators", Proceedings of the 31st International Conference on Very Large Data Bases (VLDB), pp.1097~1107 (2005)
- "Regular expression matching can be simple and fast (but is slow in Java, Perl, PHP, Python, Ruby, ...)" (2007)[URL]
- M. D. McIlroy, "Enumerating the strings of regular languages", Journal of Functional Programming, 14(5), pp.503~518 (2004)
- K. Thompson, "Programming techniques : regular expression search algorithm", Communications of the ACM, 11(6), pp.419~422 (1968)
- M. Poess, "New TPC benchmarks for decision support and web commerce", ACM SIGMOD Record, 29(4), pp.64~71 (2000)
- A. Crolotte, "Topics in Performance Evaluation, Measurement and Characterization", pp.137~145 (2012), Springer
- D. J. DeWitt, "Practical skew handling in parallel joins", Proceedings of the 31st International Conference on Very Large Data Bases (VLDB), pp.27~40 (1992)
- E. Lo, "MyBenchmark: generating databases for query workloads", The VLDB Journal, 23, pp.895~913 (2014)
- M. O. Rabin, "Finite automata and their decision problems", IBM Journal of Research and Development, 3(2), pp.114~125 (1959)
- M. Cognetta, "Incremental computation of infix probabilities for probabilistic finite automata", Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pp.2732~2741 (2018)
- "ReverseRegex: use regular expressions to generate text strings" (2020)[URL]
- H. Ping, "Datasynthesizer: privacy-preserving synthetic datasets", Proceedings of the 29th International Conference on Scientific and Statistical Database Management, pp.1~5 (2017)
- J. Drechsler, "Privacy in Statistical Databases", pp.148~161 (2010), Springer
- G. Caiola, "Random forests for generating partially synthetic, categorical data", Transactions on Data Privacy, 3(1), pp.27~42 (2010)
- X. Wu, "Privacy preserving database generation for database application testing", Fundamenta Informaticae, 78(4), pp.595~612 (2007)
- J. Zhang, "Privbayes: private data release via Bayesian networks", Proceedings of International Conference on Management of Data (SIGMOD), pp.1423~1434 (2014)
- N. C. Abay, "Machine Learning and Knowledge Discovery in Databases", pp.510~526 (2019), Springer