Real Scene Text Image Super-Resolution Based on Multi-Scale and Attention Fusion
0
조회
0
다운로드
0
인용
0
참고문헌
키워드
초록
Plenty of works have indicated that single image super-resolution (SISR) models relying on synthetic datasetsare difficult to be applied to real scene text image super-resolution (STISR) for its more complex degradation. The up-to-date dataset for realistic STISR is called TextZoom, while the current methods trained on this datasethave not considered the effect of multi-scale features of text images. In this paper, a multi-scale and attentionfusion model for realistic STISR is proposed. The multi-scale learning mechanism is introduced to acquiresophisticated feature representations of text images; The spatial and channel attentions are introduced to capturethe local information and inter-channel interaction information of text images; At last, this paper designs amulti-scale residual attention module by skillfully fusing multi-scale learning and attention mechanisms. Theexperiments on TextZoom demonstrate that the model proposed increases scene text recognition’s (ASTER)average recognition accuracy by 1.2% compared to text super-resolution network.
Abstract
Plenty of works have indicated that single image super-resolution (SISR) models relying on synthetic datasetsare difficult to be applied to real scene text image super-resolution (STISR) for its more complex degradation. The up-to-date dataset for realistic STISR is called TextZoom, while the current methods trained on this datasethave not considered the effect of multi-scale features of text images. In this paper, a multi-scale and attentionfusion model for realistic STISR is proposed. The multi-scale learning mechanism is introduced to acquiresophisticated feature representations of text images; The spatial and channel attentions are introduced to capturethe local information and inter-channel interaction information of text images; At last, this paper designs amulti-scale residual attention module by skillfully fusing multi-scale learning and attention mechanisms. Theexperiments on TextZoom demonstrate that the model proposed increases scene text recognition’s (ASTER)average recognition accuracy by 1.2% compared to text super-resolution network.
참고문헌 (28)
- Xiaoxue Chen, "Text Recognition in the Wild", ACM Computing Surveys, 54(2), pp.1~35 (2021), Association for Computing Machinery (ACM)[DOI]
- W. Liu, "Star-Net: a spatial attention residue network for scene text recognition", Proceedings of the British Machine Vision Conference (BMVC) (2016)
- Z. Cheng, "Focusing attention: towards accurate text recognition in natural images", Proceedings of the IEEE International Conference on Computer Vision (ICCV), pp.5086~5094 (2017)
- Canjie Luo, "MORAN: A Multi-Object Rectified Attention Network for scene text recognition", Pattern Recognition, 90, pp.109~118 (2019), Elsevier BV[DOI]
- Baoguang Shi, "ASTER: An Attentional Scene Text Recognizer with Flexible Rectification", IEEE Transactions on Pattern Analysis and Machine Intelligence, 41(9), pp.2035~2048 (2019), Institute of Electrical and Electronics Engineers (IEEE)[DOI]
- Enze Xie, "Scene Text Detection with Supervised Pyramid Context Network", Proceedings of the AAAI Conference on Artificial Intelligence, 33(01), pp.9038~9045 (2019), Association for the Advancement of Artificial Intelligence (AAAI)[DOI]
- W. Wang, "Shape robust text detection with progressive scale expansion network", Proceedings of the IEEE, 2019, 9336~9345
- W. Wang, "Efficient and accurate arbitraryshaped text detection with pixel aggregation network", Proceedings of the IEEE, 2019, 8439~8448
- J. Baek, "What is wrong with scene text recognition model comparisons? dataset and model analysis", Proceedings of the IEEE, 2019, 4714~4722
- Chao Dong, "Image Super-Resolution Using Deep Convolutional Networks", IEEE Transactions on Pattern Analysis and Machine Intelligence, 38(2), pp.295~307 (2016), Institute of Electrical and Electronics Engineers (IEEE)[DOI]
- J. Kim, "Accurate image super-resolution using very deep convolutional networks", Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp.1646~1654 (2016)
- B. Lim, "Enhanced deep residual networks for single image superresolution", Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp.1132~1140 (2017)
- Buzhong Liu, "Lightweight Single Image Super-Resolution by Channel Split Residual Convolution", JIPS(Journal of Information Processing Systems), 18(1), pp.12~25 (2022), 한국정보처리학회[DOI]
- Y. Zhang, "Residual dense network for image super-resolution", Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp.2472~2481 (2018)
- W. Wang, "Computer Vision–ECCV 2020", pp.650~666 (2020), Springer
- Chuantao Fang, "TSRGAN: Real-world text image super-resolution based on adversarial learning and triplet attention", Neurocomputing, 455, pp.88~96 (2021), Elsevier BV[DOI]
- W. S. Lai, "Deep Laplacian pyramid networks for fast and accurate super-resolution", Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp.5835~5843 (2017)
- C. Szegedy, "Rethinking the inception architecture for computer vision", Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp.2818~2826 (2016)
- Jinghui Qin, "Multi-scale feature fusion residual network for Single Image Super-Resolution", Neurocomputing, 379, pp.334~342 (2020), Elsevier BV[DOI]
- S. Woo, "CBAM: convolutional block attention module", Proceedings of the European Conference on Computer Vision (ECCV), pp.3~19 (2018)
- K. He, "Deep residual learning for image recognition", Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp.770~778 (2016)
- T. Tong, "Image super-resolution using dense skip connections", Proceedings of the IEEE International Conference on Computer Vision, pp.4809~4817 (2017)
- Y. Zhang, "Image super-resolution using very deep residual channel attention networks", Proceedings of the European Conference on Computer Vision (ECCV), pp.294~310 (2018)
- Baoguang Shi, "An End-to-End Trainable Neural Network for Image-Based Sequence Recognition and Its Application to Scene Text Recognition", IEEE Transactions on Pattern Analysis and Machine Intelligence, 39(11), pp.2298~2304 (2017), Institute of Electrical and Electronics Engineers (IEEE)[DOI]
- M. Jaderberg, "Spatial transformer networks", Advances in Neural Information Processing Systems, 28, pp.2017~2025 (2015)
- W. Shi, "Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network", Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp.1874~1883 (2016)
- J. W. Liu, "Research progress on batch normalization of deep learning and its related algorithms", Acta Automatica Sinica, 46(6), pp.1090~1120 (2020)
- C. Ledig, "Photo-realistic single image super-resolution using a generative adversarial network", Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp.105~114 (2017)