영상 데이터 특징 커버리지 기반 딥러닝 모델 검증 기법
Deep Learning Model Validation Method Based on Image Data Feature Coverage
0
조회
0
다운로드
2
인용
0
참고문헌
키워드
초록
딥러닝 기법은 영상 처리 분야에서 높은 성능을 입증 받아 다양한 분야에서 적용되고 있다. 이러한 딥러닝 모델의 검증에 가장 널리 사용되는방법으로는 홀드아웃 검증 방법, k-겹 교차 검증 방법, 부트스트랩 방법 등이 있다. 이러한 기존의 기법들은 데이터 셋을 분할하는 과정에서 클래스간의 비율에 대한 균형을 고려하지만, 같은 클래스 내에서도 존재하는 다양한 특징들의 비율은 고려하지 않고 있다. 이러한 특징들을 고려하지않을 경우, 일부 특징에 편향된 검증 결과를 얻게 될 수 있다. 따라서 본 논문에서는 기존 검증 방법들을 개선하여 영상 분류를 위한 데이터 특징커버리지 기반의 딥러닝 모델 검증 기법을 제안한다. 제안하는 기법은 딥러닝 모델의 학습과 검증을 위한 훈련 데이터 셋과 평가 데이터 셋이전체 데이터 셋의 특징을 얼마나 반영하고 있는지 수치로 측정할 수 있는 데이터 특징 커버리지를 제안한다. 이러한 방식은 전체 데이터 셋의특징을 모두 포함하도록 커버리지를 보장하여 데이터 셋을 분할할 수 있고, 모델의 평가 결과를 생성한 특징 군집 단위로 분석할 수 있다. 검증결과, 훈련 데이터 셋의 데이터 특징 커버리지가 낮아질 경우, 모델이 특정 특징에 편향되게 학습하여 모델의 성능이 낮아지며, Fashion-MNIST의경우 정확도가 8.9%까지 차이나는 것을 확인하였다.
Abstract
Deep learning techniques have been proven to have high performance in image processing and are applied in various fields. Themost widely used methods for validating a deep learning model include a holdout verification method, a k-fold cross verification method,and a bootstrap method. These legacy methods consider the balance of the ratio between classes in the process of dividing the dataset, but do not consider the ratio of various features that exist within the same class. If these features are not considered, verificationresults may be biased toward some features. Therefore, we propose a deep learning model validation method based on data feature coveragefor image classification by improving the legacy methods. The proposed technique proposes a data feature coverage that can be measurednumerically how much the training data set for training and validation of the deep learning model and the evaluation data set reflectsthe features of the entire data set. In this method, the data set can be divided by ensuring coverage to include all features of the entiredata set, and the evaluation result of the model can be analyzed in units of feature clusters. As a result, by providing feature clusterinformation for the evaluation result of the trained model, feature information of data that affects the trained model can be provided.
참고문헌 (32)
- A. Esteva, "Dermatologist-level classification of skin cancer with deep neural networks", Nature, 542(7639), pp.115~118 (2017)
- F. Milletari, "V-net: Fully convolutional neural networks for volumetric medical image segmentation", 2016 Fourth International Conference on 3D Vision (3DV). IEEE (2016)
- A. Angelova, "Real-time pedestrian detection with deep network cascades", ~ (2015)
- M. Bojarski, "End to end learning for self-driving cars", arXiv preprint arXiv : 1604. 07316, ~ (2016)
- A. Ferdowsi, "Deep learning for signal authentication and security in massive internet-of-things systems", IEEE Transactions on Communications, 67(2), pp.1371~1387 (2018)
- R. Kohavi, "A study of cross-validation and bootstrap for accuracy estimation and model selection", Ijcai, 14(2), pp.1137~1145 (1995)
- S. Mani, "Coverage testing of deep learning models using dataset characterization", arXiv preprint arXiv : 1911. 07309, ~ (2019)
- T. Lindeberg, "Scale invariant feature transform", 10491~ (2012)
- N. Dalal, "Histograms of oriented gradients for human detection", 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05), 1 (2005), IEEE
- R. Lienhart, "An extended set of haar-like features for rapid object detection", Proceedings. International Conference on Image Processing, 1 (2002), IEEE
- M. Flickner, "Query by image and video content : The QBIC system", Computer, 28(9), pp.23~32 (1995)
- P. Sermanet, "Overfeat : Integrated recognition, localization and detection using convolutional networks", arXiv preprint arXiv:1312.6229, ~ (2013)
- A. Sharif Razavian, "CNN features off-the-shelf: An astounding baseline for recognition" (2014), Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops
- L. Liu, "The treasure beneath convolutional layers: Cross-convolutional-layer pooling for image classification", Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2015)
- H. Xiao, "Fashion-mnist : A novel image dataset for benchmarking machine learning algorithms", arXiv preprint arXiv:1708.07747, ~ (2017)
- A. Krizhevsky, "Learning multiple layers of features from tiny images", 7~ (2009)
- S. Yadav, "Analysis of k-fold cross-validation over hold-out validation on colossal datasets for quality classification", 2016 IEEE 6th International Conference on Advanced Computing (IACC), IEEE (2016)
- Y. Xu, "On splitting training and validation set : A comparative study of cross-validation, bootstrap and systematic sampling for estimating the generalization performance of supervised learning", Journal of Analysis and Testing, 2(3), pp.249~262 (2018)
- M. Ozuysal, "Fast keypoint recognition using random ferns", IEEE Transactions on Pattern Analysis and Machine Intelligence, 32(3), pp.448~461 (2009)
- T. Ojala, "A comparative study of texture measures with classification based on featured distributions", Pattern Recognition, 29(1), pp.51~59 (1996)
- B. Froba, "Face detection with the modified census transform", Sixth IEEE International Conference on Automatic Face and Gesture Recognition, 2004. Proceedings, IEEE (2004)
- G. Chandrashekar, "A survey on feature selection methods", Computers & Electrical Engineering, 40(1), pp.16~28 (2014)
- D. ping Tian, "A review on image feature extraction and representation techniques", International Journal of Multimedia and Ubiquitous Engineering, 8(4), pp.385~396 (2013)
- A. K. Jain, "Image retrieval using color and shape", Pattern Recognition, 29(8), pp.1233~1244 (1996)
- G. Pass, "Histogram refinement for contentbased image retrieval", Proceedings Third IEEE Workshop on Applications of Computer Vision, WACV'96. IEEE (1996)
- J. Huang, "Image indexing using color correlograms", Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition, IEEE (1997)
- L. Rokach, "Data Mining and Knowledge Discovery Handbook", pp.321~352 (2005), Springer
- S. Na, "Research on k-means clustering algorithm : An improved k-means clustering algorithm", 2010 Third International Symposium on intelligent Information Technology and Security Informatics, IEEE (2010)
- R. Xu, "Survey of clustering algorithms", IEEE Transactions on Neural Networks, 16(3), pp.645~678 (2005)
- R. Bro, "Principal component analysis", Analytical Methods, 6(9), pp.2812~2831 (2014)
- J. Xue, "Using principal components analysis(PCA)with cluster analysis to study the organic geochemistry of sinking particles in the ocean", Organic Geochemistry, 42(4), pp.356~367 (2011)
- T. M. Kodinariya, "Review on determining number of Cluster in K-Means Clustering", International Journal, 1(6), pp.90~95 (2013)