Image reconstruction technique using deep learning architecture
1
조회
0
다운로드
2
인용
0
참고문헌
키워드
초록
Existing frame interpolation techniques require a large amount of training data because they use models to extract general motions from a large number of videos. Using these models, they estimate a motion vector for the next frame using a motion vector from the previous frame of the same video; this causes poor accuracy, characterized by blurred or distorted images, because the cumulative error increases with the learning time and the number of generated frames. Therefore, it is necessary to construct a robust model, by simplifying the learning and generation models, which reduces computation and generates only the changed regions of the intermediate frames. In order to realize a simplified model that generates pixels in changed regions, we constructed a motion feature extraction and learning model based on convolution neural network (CNN) and recurrent neural network (RNN). In addition, we propose an algorithm which used a motion probability map to generate intermediate frames via deconvolution layers for regions where motion occurs. To train and verify the proposed model, we performed experiments based on actual videos from a dataset created by capturing the motions of a robot.
Abstract
Existing frame interpolation techniques require a large amount of training data because they use models to extract general motions from a large number of videos. Using these models, they estimate a motion vector for the next frame using a motion vector from the previous frame of the same video; this causes poor accuracy, characterized by blurred or distorted images, because the cumulative error increases with the learning time and the number of generated frames. Therefore, it is necessary to construct a robust model, by simplifying the learning and generation models, which reduces computation and generates only the changed regions of the intermediate frames. In order to realize a simplified model that generates pixels in changed regions, we constructed a motion feature extraction and learning model based on convolution neural network (CNN) and recurrent neural network (RNN). In addition, we propose an algorithm which used a motion probability map to generate intermediate frames via deconvolution layers for regions where motion occurs. To train and verify the proposed model, we performed experiments based on actual videos from a dataset created by capturing the motions of a robot.
참고문헌 (12)
- J. H. Seong, "Environment adaptive localization method using Wi-Fi and bluetooth low energy", Wireless Personal Communications, pp.1~14 (2017)
- J. H. Seong, "High-speed positioning and automatic updating techni- que using Wi-Fi and UWB in a ship", Wireless Personal Communications, 94(3), pp.1105~1121 (2017)[DOI]
- 김원열, "Welding and weaving speed estimation using acceleration sensor based on SVM", 한국마린엔지니어링학회지, 41(10), pp.1018~1023 (2017), 한국마린엔지니어링학회[DOI]
- 이준희, "객체 검출 기반 이미지 캡션 알고리즘 연구", 한국마린엔지니어링학회지, 41(7), pp.683~689 (2017), 한국마린엔지니어링학회[DOI]
- J. Revaud, "Epicflow: Edge-preserving interpolation of correspondences for optical flow", Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp.1164~1172 (2015)
- S. Meyer, "Phase-based frame interpolation for video", Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp.1410~1418 (2015)
- D. S. Choi, "MAP-based motion refinement algorithm for block-based motion-compensated frame interpolation", IEEE Transactions on Circuits and Systems for Video Technology, 26(10), pp.1789~1804 (2016)[DOI]
- D. Rufenacht, "Temporally consistent high frame-rate upsampling with motion sparsification", Multimedia Signal Processing (MMSP), 2016 IEEE 18th International Workshop on, pp.1~6 (2016), IEEE
- D. Guo, "Motion-compensated frame inter- polation with weighted motion estimation and hier- archical vector refinement", Neurocomputing, 181(C), pp.76~85 (2016)[DOI]
- G. Long, "Learning image matching by simply watching video", European Conference on Computer Vision (2016)
- S. Niklaus, "Video frame interpolation via adaptive convolution", CVPR 2017 (2017)
- M. Mathieu, "Deep multi-scale video prediction beyond mean square error", ~ (2015)