Document Image Binarization by GAN with Unpaired Data Training
1
조회
0
다운로드
1
인용
0
참고문헌
키워드
초록
Data is critical in deep learning but the scarcity of data often occurs in research, especially in the preparation of the paired training data. In this paper, document image binarization with unpaired data is studied by introducing adversarial learning, excluding the need for supervised or labeled datasets. However, the simple extension of the previous unpaired training to binarization inevitably leads to poor performance compared to paired data training. Thus, a new deep learning approach is proposed by introducing a multi-diversity of higher quality generated images. In this paper, a two-stage model is proposed that comprises the generative adversarial network (GAN) followed by the U-net network. In the first stage, the GAN uses the unpaired image data to create paired image data. With the second stage, the generated paired image data are passed through the U-net network for binarization. Thus, the trained U-net becomes the binarization model during the testing. The proposed model has been evaluated over the publicly available DIBCO dataset and it outperforms other techniques on unpaired training data. The paper shows the potential of using unpaired data for binarization, for the first time in the literature, which can be further improved to replace paired data training for binarization in the future.
Abstract
Data is critical in deep learning but the scarcity of data often occurs in research, especially in the preparation of the paired training data. In this paper, document image binarization with unpaired data is studied by introducing adversarial learning, excluding the need for supervised or labeled datasets. However, the simple extension of the previous unpaired training to binarization inevitably leads to poor performance compared to paired data training. Thus, a new deep learning approach is proposed by introducing a multi-diversity of higher quality generated images. In this paper, a two-stage model is proposed that comprises the generative adversarial network (GAN) followed by the U-net network. In the first stage, the GAN uses the unpaired image data to create paired image data. With the second stage, the generated paired image data are passed through the U-net network for binarization. Thus, the trained U-net becomes the binarization model during the testing. The proposed model has been evaluated over the publicly available DIBCO dataset and it outperforms other techniques on unpaired training data. The paper shows the potential of using unpaired data for binarization, for the first time in the literature, which can be further improved to replace paired data training for binarization in the future.
참고문헌 (37)
- Simon M. Lucas, "ICDAR 2003 robust reading competitions: entries, results, and future directions", International Journal of Document Analysis and Recognition (IJDAR), 7(2-3), pp.105~122 (2005), Springer Science and Business Media LLC[DOI]
- N. Stamatopoulos, "ICDAR 2013 Handwriting Segmentation Contest", 12th International Conference on Document Analysis and Recognition (ICDAR), pp.1402~1406 (2013)
- Loannis Pratikakis, "ICDAR 2017 Competition on Document Image Binarization (DIBCO 2017)", 14th IAPR International Conference on Document Analysis and Recognition (2017)
- C.Tensmeyer, "Document Image Binarization with Fully Convolutional Neural networks", 14th IAPR International Conference on Document Analysis and Recognition (2017)
- Quang Nhat Vo, "Binarization of degraded document images based on hierarchical deep supervised network", Pattern Recognition, 74(C), pp.568~586 (2018), Elsevier BV[DOI]
- Jorge Calvo-Zaragoza, "A selectional auto-encoder approach for document image binarization", Journal Pattern Recognition, 86 (2017)[DOI]
- O.Ronneberger, "International Conference on Medical Image Computing and computer-assisted Intervention", pp.234~241 (2015), Springer
- J. Zhu, "Unpaired Image-to-image Translation Using Cycle-Consistent Adversarial Networks", ICCV (2017)
- A. K. Bhunia, "Improving Document Binarization Via Adversarial Noise-Texture Augmentation", ICIP (2019)
- I. J. Goodfellow, "Generative adversarial nets", NIPS'14 Proceedings of the 27th International Conference on Neural Information Processing Systems, 2, pp.2672~2680 (2014)
- E. L. Denton, "Deep generative image models using a laplacian pyramid of adversarial networks", NIPS (2015)
- A. Radford, "Unsupervised representation learning with deep convolutional generative adversarial networks", ICLR (2016)
- M. Mirza, "Conditional generative adversarial nets", arXiv preprint arXiv : 1411. 1784, ~ (2014)
- G. Perarnau, "Invertible conditional gans for´image editing", NIPS Workshop (2016)
- E. Mansimov, "Generating images from captions with attention", ICLR (2015)
- H. Liu, "Coherent semantic attention for image inpainting", Proc. IEEE, 2019, 4170~4179
- P. Isola, "Image-to-image translation with conditional adversarial networks", CVPR (2017)
- J. Y. Zhu, "Toward multimodal imageto-image translation", NIPS (2017)
- X. Chen, "Infogan: Interpretable representation learning by information maximizing generative adversarial nets", NIPS (2016)
- I. Higgins, "beta-vae:Learning basic visual concepts with a constrained variational framework", ICLR (2017)
- X. Huang, "Multimodal unsupervised image-to-image translation", ECCV (2018)
- L. A. Gatys, "Image style transfer using convolutional neural networks", CVPR, pp.2414~2423 (2016)
- J. Johnson, "Perceptual losses for real-time style transfer and super-resolution", ECCV (2016)
- D. Ulyanov, "Texture networks: Feed-forward synthesis of textures and stylized images", ICML (2016)
- L. A. Gatys, "Preserving color in neural artistic style transfer", preprint arXiv:1606.05897, ~ (2016)
- B. Gatos, "ICDAR 2009 document image binarization contest(DIBCO 2009)", ICDAR, pp.1375~1382 (2009)
- I. Pratikakis, "ICDAR 2011 document image binarization contest(DIBCO 2011)", ICDAR, pp.1506~1510 (2011)
- I. Pratikakis, "ICDAR 2013 document image binarization contest(DIBCO 2013)", ICDAR, pp.1471~1476 (2013)
- I. Pratikakis, "H-DIBCO 2010-handwritten document image binarization competition", ICFHR, pp.727~732 (2010)
- I. Pratikakis, "ICFHR 2012 competition on handwritten document image binarization(H-DIBCO 2012)", ICFHR, pp.817~822 (2012)
- I. Pratikakis, "ICFHR 2014 competition on handwritten document image binarization(H-DIBCO 2014)", ICFHR, pp.809~813 (2014)
- F. Deng, "BinarizationShop : a user-assisted software suite for converting old documents to black-and-white", Proc. Annu. Joint Conf. Digit. Libraries, pp.255~258 (2010)
- H. Z. Nafchi, "An efficient ground-truthing tool for binarization of historical manuscripts", ICDAR, pp.807~811 (2013)
- R. Hedjam, "ICDAR 2015 multispectral text extraction contest(MS-TEx 2015)", ICDAR, pp.1181~1185 (2015)
- B. Gatos, "ICDAR 2009 document image binarization contest (DIBCO 2009)", Document Analysis and Recognition, 2009, ICDAR’09. 10th International Conference, pp.1375~1382 (2009), IEEE
- K. Ntirogiannis, "Performance Evaluation Methodology for Historical Document Image Binarization", IEEE Transactions on Image Processing, 22(2), pp.595~609 (2013)
- L. Gatys, "Texture synthesis using convolutional neural networks", NIPS, pp.262~270 (2015)