Improvement of RocksDB Performance via Large-Scale Parameter Analysis and Optimization
2
조회
0
다운로드
0
인용
0
참고문헌
키워드
초록
Database systems usually have many parameters that must be configured by database administrators and users. RocksDB achieves fast data writing performance using a log-structured merged tree. This database has manyparameters associated with write and space amplifications. Write amplification degrades the databaseperformance, and space amplification leads to an increased storage space owing to the storage of unwanteddata. Previously, it was proven that significant performance improvements can be achieved by tuning thedatabase parameters. However, tuning the multiple parameters of a database is a laborious task owing to thelarge number of potential configuration combinations. To address this problem, we selected the importantparameters that affect the performance of RocksDB using random forest. We then analyzed the effects of theselected parameters on write and space amplifications using analysis of variance. We used a genetic algorithmto obtain optimized values of the major parameters. The experimental results indicate an insignificant reduction(-5.64%) in the execution time when using these optimized values; however, write amplification, spaceamplification, and data processing rates improved considerably by 20.65%, 54.50%, and 89.68%, respectively,as compared to the performance when using the default settings.
Abstract
Database systems usually have many parameters that must be configured by database administrators and users. RocksDB achieves fast data writing performance using a log-structured merged tree. This database has manyparameters associated with write and space amplifications. Write amplification degrades the databaseperformance, and space amplification leads to an increased storage space owing to the storage of unwanteddata. Previously, it was proven that significant performance improvements can be achieved by tuning thedatabase parameters. However, tuning the multiple parameters of a database is a laborious task owing to thelarge number of potential configuration combinations. To address this problem, we selected the importantparameters that affect the performance of RocksDB using random forest. We then analyzed the effects of theselected parameters on write and space amplifications using analysis of variance. We used a genetic algorithmto obtain optimized values of the major parameters. The experimental results indicate an insignificant reduction(-5.64%) in the execution time when using these optimized values; however, write amplification, spaceamplification, and data processing rates improved considerably by 20.65%, 54.50%, and 89.68%, respectively,as compared to the performance when using the default settings.
참고문헌 (21)
- F. Mei, "LSM-tree managed storage for large-scale key-value store", IEEE Transactions on Parallel and Distributed Systems, 30(2), pp.400~414 (2018)
- "A persistent key-value store for fast storage environments" (2021)[URL]
- P. O’Neil, "The log-structured merge-tree(LSM-tree)", Acta Informatica, 33(4), pp.351~385 (1996)
- K. Ouaknine, "Optimization of RocksDB for Redis on flash", Proceedings of the International Conference on Compute and Data Analysis, pp.155~161 (2017)
- Z. Cao, "Characterizing, modeling, and benchmarking RocksDB keyvalue workloads at Facebook", Proceedings of the 18th USENIX Conference on File and Storage Technologies (FAST 2020), pp.209~223 (2020)
- H. Kim, "Optimizing RocksDB for better read throughput in blockchain systems", Proceedings of the 23rd International Computer Science and Engineering Conference (ICSEC), pp.305~309 (2019)
- X. Y. Hu, "Write amplification analysis in flash-based solid state driv", Proceedings of SYSTOR 2009: The Israeli Experimental Systems Conference, pp.1~9 (2009)
- Y. Lu, "Extending the lifetime of flash-based storage through reducing write amplification from file systems", Proceedings of the 11th USENIX Conference on File and Storage Technologies (FAST 2013), pp.257~270 (2013)
- S. Odeh, "NAND flash architectures reducing write amplification through multi-write codes", Proceedings of 2014 30th Symposium on Mass Storage Systems and Technologies (MSST), pp.1~10 (2014)
- L. Breiman, "Random forests", Machine Learning, 45(1), pp.5~32 (2001)
- D. C. Howell, "Statistical Methods for Psychology" (2002), Duxbury
- D. Whitley, "A genetic algorithm tutorial", Statistics and Computing, 4(2), pp.65~85 (1994)
- S. J. Mardle, "An investigation of genetic algorithms for the optimization of multiobjective fisheries bioeconomic models", International Transactions in Operational Research, 7(1), pp.33~49 (2000)
- S. Jena, "Multi-objective optimization of design parameters of a shell & tube type heat exchanger using genetic algorithm", International Journal of Current Engineering and Technology, 3(4), pp.1379~1386 (2013)
- "Benchmarking tools" (2022)[URL]
- S. Dong, "Optimizing space amplification in RocksDB", Proceedings of the 8th Biennial Conference on Innovative Data Systems Research (CIDR) (2017)
- F. Pedregosa, "Scikit-learn : machine learning in Python", Journal of Machine Learning Research, 12, pp.2825~2830 (2011)
- K. Kanellis, "Too many knobs to tune? towards faster database tuning by pre-selecting important knobs", Proceedings of the 12th USENIX Workshop on Hot Topics in Storage and File Systems (HotStorage 2020) (2020)
- D. Van Aken, "Automatic database management system tuning through large-scale machine learning", Proceedings of the 2017 ACM International Conference on Management of Data, pp.1009~1024 (2017)
- Y. Zhu, "BestConfig: tapping the performance potential of systems via automatic configuration tuning", Proceedings of the 2017 Symposium on Cloud Computing, pp.338~350 (2017)
- J. Zhang, "An end-to-end automatic cloud database tuning system using deep reinforcement learning", Proceedings of the 2019 International Conference on Management of Data, pp.415~432 (2019)