Finding the best suited autoencoder for reducing model complexity
0
조회
0
다운로드
0
인용
0
참고문헌
키워드
초록
Basically, machine learning models use input data to produce results. Sometimes, the input data is too complicated for the models to learn useful patterns. Therefore, feature engineering is a crucial data preprocessing step for constructing a proper feature set to improve the performance of such models. One of the most efficient methods for automating feature engineering is the autoencoder, which transforms the data from its original space into a latent space. However certain factors, including the datasets, the machine learning models, and the number of dimensions of the latent space (denoted by k), should be carefully considered when using the autoencoder. In this study, we design a framework to compare two data preprocessing approaches: with and without autoencoder and to observe the impact of these factors on autoencoder. We then conduct experiments using autoencoders with classifiers on popular datasets. The empirical results provide a perspective regarding the best suited autoencoder for these factors.
Abstract
Basically, machine learning models use input data to produce results. Sometimes, the input data is too complicated for the models to learn useful patterns. Therefore, feature engineering is a crucial data preprocessing step for constructing a proper feature set to improve the performance of such models. One of the most efficient methods for automating feature engineering is the autoencoder, which transforms the data from its original space into a latent space. However certain factors, including the datasets, the machine learning models, and the number of dimensions of the latent space (denoted by k), should be carefully considered when using the autoencoder. In this study, we design a framework to compare two data preprocessing approaches: with and without autoencoder and to observe the impact of these factors on autoencoder. We then conduct experiments using autoencoders with classifiers on popular datasets. The empirical results provide a perspective regarding the best suited autoencoder for these factors.
참고문헌 (32)
- Y. Bengio, "Representation learning : A review and new perspectives", IEEE transactions on pattern analysis and machine intelligence, 35(8), pp.1798~1828 (2013)
- B. Ghojogh, "Feature selection and feature extraction in pattern analysis: A literature review", arXiv preprint arXiv:1905.02845, ~ (2019)
- M. Tschannen, "Recent advances in autoencoder-based representation learning", arXiv preprint arXiv:1812.05069, ~ (2018)
- I. Goodfellow, "Deep Learning" (2016), MIT Press
- P. Vincent, "Extracting and composing robust features with denoising autoencoders", Proceedings of the 25th International Conference on Machine Learning, pp.1096~1103 (2008)
- K. Sohn, "Learning Structured Output Representation Using Deep Conditional Generative Models", Proceedings of the 28th International Conference on Neural Information Processing Systems, 2, pp.3483~3491 (2015)
- J. Walker, "An Uncertain Future: Forecasting from Static Images Using Variational Autoencoders", eprint arXiv:1606.07873, ~ (2016)
- Thanh-Vu Dang, "Removing Out – Of – Distribution Samples on Classification Task", 스마트미디어저널, 9(3), pp.80~89 (2020), (사)한국스마트미디어학회[DOI]
- D. P. Kingma, "Auto-encoding variational bayes", arXiv preprint arXiv:1312.6114, ~ (2013)
- C. Doersch, "Tutorial on variational autoencoders", arXiv preprint arXiv:1606.05908, ~ (2016)
- H. Kim, "Disentangling by Factorising", Proceedings of the 35th International Conference on Machine Learning, 80, pp.2649~2658 (2018)
- R. T. Q. Chen, "Isolating Sources of Disentanglement in Variational Autoencoders", NIPS'18 : Proceedings of the 32nd International Conference on Neural Information Processing Systems, pp.2615~2625 (2018)
- S. Zhao, "Infovae : Information maximizing variational autoencoders", arXiv preprint arXiv:1706.02262, ~ (2017)
- A. Gretton, "A Kernel Method for the Two-Sample-Problem", Advances in Neural Information Processing Systems, 19, pp.513~520 (2006)
- A. Makhzani, "Adversarial autoencoders", arXiv preprint arXiv:1511.05644, ~ (2015)
- B. Gayathri, "An automated technique using Gaussian Naive Bayes classifier to classify breast cancer", International Journal of Computer Applications, 148(6), pp.16~21 (2016)
- C. Cortes, "Support-vector networks", Machine learning, 20(3), pp.273~297 (1995)
- G. James, "An introduction to statistical learning" (2013), Springer
- L. Breiman, "Random forests", Machine learning, 45(1), pp.5~32 (2001)
- Y. LeCun, "Gradient-based learning applied to document recognition", Proceedings of the IEEE, 86(11), pp.2278~2324 (1998)
- D. P. Kingma, "Adam : A Method for Stochastic Optimization", 3rd International Conference on Learning Representations(ICLR) (2015)
- Y. LeCun, "MNIST handwritten digit database" (2010), ATT Labs
- H. Xiao, "Fashion-MNIST : a Novel Image Dataset for Benchmarking Machine Learning Algorithms", CoRR, ~ (2017)
- A. Krizhevsky, "Learning multiple layers of features from tiny images", ~ (2009)
- Y. Bengio, "Deep Generative Stochastic Networks Trainable by Backprop", ICML'14 : Proceedings of the 31st International Conference on International Conference on Machine Learning, 32, pp.226~234 (2014)
- T. Salimans, "Markov Chain Monte Carlo and Variational Inference : Bridging the Gap", ICML'15 : Proceedings of the 32nd International Conference on International Conference on Machine Learning, 37, pp.1218~1226 (2015)
- T. D. Kulkarni, "Deep Convolutional Inverse Graphics Network", NIPS'15 : Proceedings of the 28th International Conference on Neural Information Processing Systems, 2, pp.2539~2547 (2015)
- D. J. Rezende, "Stochastic Backpropagation and Approximate Inference in Deep Generative Models", Proceedings of the 31st International Conference on Machine Learning, 32(2), pp.1278~1286 (2014)
- K. Gregor, "DRAW : A Recurrent Neural Network For Image Generation", Proceedings of Machine Learning Research, 37, pp.1462~1471 (2015)
- D. P. Kingma, "Semi-Supervised Learning with Deep Generative Models", NIPS'14 : Proceedings of the 27th International Conference on Neural Information Processing Systems, 2, pp.3581~3589 (2014)
- Sudarshan Pant, "A Fall Detection Technique using Features from Multiple Sliding Windows", 스마트미디어저널, 7(4), pp.79~89 (2018), (사)한국스마트미디어학회[DOI]
- Tien Duong Vu, "Classifying Instantaneous Cognitive States from fMRI using Discriminant based Feature Selection and Adaboost", 스마트미디어저널, 5(1), pp.30~37 (2016), (사)한국스마트미디어학회