시각화 오토인코더를 사용한 학습된 심층신경망의 잠재공간 조작 시스템
A Method for Latent Space Control System of a Learned Deep Neural Network Using a Visualization Autoencoder
0
조회
0
다운로드
0
인용
0
참고문헌
키워드
초록
최근에 생성모델, 특히 ChatGPT와 DALLE-3과 같은 생성모델들은 사회에 큰 영향을 미치며, 이러한 모델들의 잠재공간을 정교하게 조작하고 이해하는 기술이 중요해지고 있다. 본 논문에서는 학습된 생성모델의 잠재공간을 시각화하여 조작하는 새로운 방법인 “시각화 오토인코더” 방법을 제안한다. 이 방법은 생성모델의 잠재벡터를 입력으로 하여 3차원 이하의 시각화 가능한 차원으로 축소하여조작하는 것을 목표로 한다. 본 연구에서는 생성모델의 잠재공간 조작을 “관심 있는 샘플 생성”과 “샘플 간 변환 과정 시각화” 두 가지로 정의하고, 이를 위해 필요한 수학적 성질 “일대일 대응”과 “locally smoothness”를 가정한 심층신경망 기반의 생성모델로 제한한다. MNIST 데이터셋을 사용하여 실험한 결과 이러한 성질을 만족하는 심층신경망 모델에 대해 제안된 시각화 오토인코더 방법으로 관심있는 샘플을 생성, 변환 과정을 시각화할 수 있음을 확인하였다.
Abstract
Recent generative models, especially generative models such as ChatGPT and DALLE-3, have had a great impact on society, and technology to intricately manipulate and understand the latent space of these models is becoming important. In this paper, we propose the “Visualization Autoencoder” method, a new method that visualizes and manipulates the latent space of the learned generative model. This method aims to manipulate the latent vectors of the generative model by reducing them to three or less dimensions that can be visualized. In this study, the latent space manipulation of the generative model is defined as two types: “generating samples of interest” and “visualizing the transformation process between samples”. For this purpose, we limit ourselves to a deep neural network-based generative model that assumes the necessary mathematical properties “one-to-one correspondence” and “locally smoothness”. As a result of experiments using the MNIST dataset, we confirmed that the proposed visualization autoencoder method for a deep neural network model that satisfies these properties can generate samples of interest and visualize the conversion process.
참고문헌 (28)
- X. Ren, "Learning Disentangled Representation by Exploiting Pretrained Generative Models : A Contrastive Learning View", ~ (2021)
- J. Lezama, "Overcoming the Disentanglement vs Reconstruction Trade-off via Jacobian Supervision", International Conference on Learning Representations (2019)
- W. Peebles, "The Hessian Penalty : A Weak Prior for Unsupervised Disentanglement", Computer Vision–ECCV 2020: 16th European Conference, 12351, pp.581~597 (2020)
- M. Connor, "Representing Closed Transformation Paths in Encoded Network Latent Space", Proceedings of the AAAI Conference on Artificial Intelligence, 34(04) (2020)
- K. H. Min, "Algorithmic Bias Mitigation Method via Controlling Latent Space regards to Protected Feature", Proceedings of the Korea Conference on Software Engineering, pp.491~493 (2019)
- 김용길, "이미지 생성을 위한 변동 자동 인코더 분산 제약", 한국인터넷방송통신학회 논문지, 23(3), pp.91~97 (2023), 한국인터넷방송통신학회[DOI]
- H. Kim, "Disentangling by Factorising, nternational Conference on Machine Learning", PMLR, 80, pp.2649~2658 (2018)
- T. Möllenhoff a, "Flat Metric Minimization with Applications in Generative Modeling", International Conference on Machine Learning, 97, pp.4626~4635 (2019), PMLR
- D. P. Kingma, "Glow : Generative Flow with Invertible 1x1 Convolutions", Advances in Neural Information Processing Systems, 31 (2018)
- C-H. Pham, "PCA-AE : Principal Component Analysis Autoencoder for Organising the Latent Space of Generative Networks", Journal of Mathematical Imaging and Vision, 64, pp.569~585 (2022)
- I. Higgins, "beta-VAE: Learning Basic Visual Concepts with a Constrained Variational Framework", International Conference on Learning Representations (2017)
- T. Karras, "A Style-Based Generator Architecture for Generative Adversarial Networks", Proceedings of the IEEE, 2019, 4401~4410
- T. Park, "Swapping autoencoder for deep image manipulation", Advances in Neural Information Processing Systems, 33, pp.7198~7211 (2020)
- J. H. Lee, "Attribute Preserving Face De-identification Method Using Latent Space Manipulation", Proceedings of the Korean Society of Precision Engineering Conference, pp.596~596 (2022)
- J. Zhu, "Low-RankSubspaces in GANs", Advances in Neural Information Processing Systems, 34 (2021)
- Y. Shen, "Closed-Form Factorization of Latent Semantics in GANs", Proceedings of the IEEE, 2021, 1532~1540
- H. Yang, "Discovering Interpretable Latent Space Directions of GANs Beyond Binary Attributes", Proceedings of the IEEE, 2021, 12177~12185
- A. Jahanian, "On the “steerability”of generative adversarial networks", ~ (2019)
- O. K. Yüksel, "LatentCLR: A Contrastive Learning Approach for Unsupervised Discovery of Interpretable Directions", Proceedings of the IEEE, 2021, 14263~14272
- J. Choi, "Do Not Escape From the Manifold : Discovering the Local Coordinates on the Latent Space of GANs", ~ (2021)
- A. Voynov, "Unsupervised Discovery of Interpretable Directions in the GAN Latent Space", International conference on machine learning, 119, pp.9786~9796 (2020), PMLR
- N. Spingarn, "GAN “Steerability” without optimization", International Conference on Learning Representations (2020)
- E. Härkönen, "GANSpace : Discovering Interpretable GAN Controls", Advances in Neural Information Processing Systems, 33 (2020)
- O. Patashnik, "StyleCLIP: Text-Driven Manipulation of StyleGAN Imagery", Proceedings of the IEEE, 2021, 2085~2094
- W. Nie, "Controllable and Compositional Generation with Latent-Space Energy-Based Models", Advances in Neural Information Processing Systems, 34 (2021)
- Z. Wu, "StyleSpace Analysis: Disentangled Controls for StyleGAN Image Generation", Proceedings of the IEEE, 2021, 12863~12872
- T. Karras, S. Laine, M. Aittala, J. Hellsten, J. Lehtinen, and T. Aila, "Analyzing and Improving the Image Quality of StyleGAN", Proceedings of the IEEE, 2020, 8110~8119
- 고영민, "파라메트릭 활성함수를 이용한 기울기 소실 문제의 완화", 정보처리학회 논문지, 10(10), pp.407~420 (2021), 한국정보처리학회