Noise2Atom: unsupervised denoising for scanning transmission electron microscopy images
2
조회
0
다운로드
0
인용
0
참고문헌
초록
We propose an effective deep learning model to denoise scanning transmission electron microscopy (STEM) image series, named Noise2Atom, to map images from a source domain S S to a target domain C C , where S S is for our noisy experimental dataset, and C C is for the desired clear atomic images. Noise2Atom uses two external networks to apply additional constraints from the domain knowledge. This model requires no signal prior, no noise model estimation, and no paired training images. The only assumption is that the inputs are acquired with identical experimental configurations. To evaluate the restoration performance of our model, as it is impossible to obtain ground truth for our experimental dataset, we propose consecutive structural similarity (CSS) for image quality assessment, based on the fact that the structures remain much the same as the previous frame(s) within small scan intervals. We demonstrate the superiority of our model by providing evaluation in terms of CSS and visual quality on different experimental datasets.
Abstract
We propose an effective deep learning model to denoise scanning transmission electron microscopy (STEM) image series, named Noise2Atom, to map images from a source domain S S to a target domain C C , where S S is for our noisy experimental dataset, and C C is for the desired clear atomic images. Noise2Atom uses two external networks to apply additional constraints from the domain knowledge. This model requires no signal prior, no noise model estimation, and no paired training images. The only assumption is that the inputs are acquired with identical experimental configurations. To evaluate the restoration performance of our model, as it is impossible to obtain ground truth for our experimental dataset, we propose consecutive structural similarity (CSS) for image quality assessment, based on the fact that the structures remain much the same as the previous frame(s) within small scan intervals. We demonstrate the superiority of our model by providing evaluation in terms of CSS and visual quality on different experimental datasets.
참고문헌 (40)
- M. Arjovsky, "Wasserstein GAN", arXiv:1701.07875, ~ (2017)
- J. Batson, "Noise2Self: Blind Denoising by Self-Supervision", arXiv:1901.11365, ~ (2019)
- T.-O. Buchholz, "Cryo-CARE: Content-Aware Image Restoration for Cryo-Transmission Electron Microscopy Data", 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019), pp.502~506 (2019), IEEE
- T. -O. Buchholz, "Content-aware image restoration for electron microscopy", Methods Cell Biol, 152, pp.277~289 (2019)[DOI]
- K. Cao, "Comparison of atomic scale dynamics for the middle and late transition metal nanocatalysts", Nat. Commun, 9(1), pp.1~10 (2018), Nature Publishing Group
- Y. Chang, "HSI-DeNet : Hyperspectral Image Restoration via Convolutional Neural Network", IEEE Trans. Geosci. Remote Sens, 57(2), pp.667~682 (2019)[DOI]
- F. Chollet, "Xception: Deep Learning with Depthwise Separable Convolutions", 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp.1800~1807 (2017), IEEE
- C. Dwyer, "Measurement of effective source distribution and its importance for quantitative interpretation of STEM images", Ultramicroscopy, 110(8), pp.952~957 (2010)
- T. Furnival, "Denoising time-resolved microscopy image sequences with singular value thresholding", Ultramicroscopy, 178, pp.112~124 (2017)[DOI]
- Z. Guo, "Deep Residual Network with Sparse Feedback for Image Restoration", Appl. Sci, 8(12), pp.2417~ (2018)
- K. He, "Deep residual learning for image recognition", Proceedings of the IEEE conference on computer vision and pattern recognition, pp.770~778 (2016)
- T. R. Henninen, "The Structure of Sub-nm Platinum Clusters at Elevated Temperatures", Angewandte Chemie International Edition (2019)[DOI]
- T. R. Henninen, "Structure matters – Direct in-situ observation of cluster nucleation at atomic scale in a liquid phase. In review", ~ (2020)
- V. S. Kadimesetty, "Convolutional neural network-based robust denoising of low-dose computed tomography perfusion maps", IEEE Trans. Radiat. Plasma Med. Sci, 3(2), pp.137~152 (2018)
- D. Keller, "Formation of gold nanoparticles in a free-standing ionic liquid triggered by heat and electron irradiation", Micron, 117, pp.16~21 (2019), Elsevier
- T. Kim, "Learning to Discover Cross-Domain Relations with Generative Adversarial Networks", Proceedings of the 34th International Conference on Machine Learning, 70, pp.1857~1865 (2017), PMLR, International Convention Centre
- E. J. Kirkland, "Simulation of annular dark field STEM images using a modified multislice method", Ultramicroscopy, 23(1), pp.77~96 (1987)
- F. Kokkinos, "Iterative Joint Image Demosaicking and Denoising Using a Residual Denoising Network", IEEE Trans. Image Process, 28(8), pp.4177~4188 (2019)[DOI]
- A. Krizhevsky, "ImageNet Classification with Deep Convolutional Neural Networks", Advances in Neural Information Processing Systems, 25, pp.1097~1105 (2012), Curran Associates, Inc
- A. Krull, "Noise2void - learning denoising from single noisy images", IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2019)
- A. Krull, "Probabilistic Noise2Void: Unsupervised Content-Aware Denoising", arXiv:1906.00651, ~ (2019)
- J. Lehtinen, "Noise2noise: Learning image restoration without clean data", arXiv preprint arXiv:1803.04189, ~ (2018)
- K. Lin, "Real photographs denoising with noise domain adaptation and attentive generative adversarial network", The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops (2019)
- P. Liu, "Multi-level wavelet-CNN for image restoration", Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp.773~782 (2018)
- A. Lucas, "Using deep neural networks for inverse problems in imaging : Beyond analytical methods", IEEE Signal Process. Mag, 35, pp.20~36 (2018)
- B. Mildenhall, "Burst denoising with kernel prediction networks", Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp.2502~2510 (2018)
- N. Moran, "Noisier2Noise:Learning to Denoise from Unpaired Noisy Data", Proceedings of the IEEE, 2020, 12064~12072
- M. Prakash, "Fully Unsupervised Probabilistic Noise2Void", 2020 IEEE 17th International Symposium on Biomedical Imaging (ISBI), pp.154~158 (2020)
- M. Ran, "Denoising of 3D magnetic resonance images using a residual encoder–decoder Wasserstein generative adversarial network", Med. Image Anal, 55, pp.165~180 (2019)
- O. Ronneberger, "U-Net:Convolutional Networks for Biomedical Image Segmentation", Medical Image Computing and Computer-Assisted Intervention – MICCAI 2015, Lecture Notes in Computer Science, pp.234~241 (2015), Springer International Publishing
- Y. Song, "Dynamic Residual Dense Network for Image Denoising", Sensors, 19(17), pp.3809~ (2019)[DOI]
- Y. Su, "Multi-scale cross-path concatenation residual network for Poisson denoising", IET Image Process, 13(8), pp.1295~1303 (2019)
- D. Ulyanov, "Instance normalization: The missing ingredient for fast stylization", arXiv preprint arXiv:1607.08022, ~ (2016)
- F. Wang, "Multi-resolution convolutional neural networks for inverse problems", Sci. Rep, 10(1), pp.1~11 (2020)[DOI]
- J. Wu, "Wasserstein divergence for gans", Proceedings of the European Conference on Computer Vision (ECCV), pp.653~668 (2018)
- W. Xie, "Deep convolutional networks with residual learning for accurate spectral-spatial denoising", Neurocomputing, 312, pp.372~381 (2018)
- Z. Yi, "Dualgan: Unsupervised dual learning for image-to-image translation", Proceedings of the IEEE International Conference on Computer Vision, pp.2849~2857 (2017)
- K. Zhang, "FFDNet : Toward a Fast and Flexible Solution for CNN based Image Denoising", IEEE Trans. Image Process, 27(9), pp.4608~4622 (2018)[DOI]
- Y. Zhang, "A poisson-gaussian denoising dataset with real fluorescence microscopy images", Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp.11710~11718 (2019)
- J. -Y. Zhu, "Unpaired image-to-image translation using cycle-consistent adversarial networks", Proceedings of the IEEE International Conference on Computer Vision, pp.2223~2232 (2017)