Edge feature를 고려한 RGB2TIR 변환 모델을 이용한 열화상 이미지 기반 Panoptic Segmentation

Abstract

In robotics and autonomous driving, accurately perceiving the environment is crucial for safety. Recent advances in panoptic segmentation, which can perform tasks such as object tracking and dynamic object removal within a single framework, are significantly enhancing the development of safer and more reliable robots and autonomous vehicles. However, most research has been limited to the RGB domain, which is vulnerable to poor lighting and adverse weather conditions. This paper proposes overcoming these limitations by introducing a robust thermal image-based panoptic segmentation model. More specifically, by leveraging RGB to thermal infrared image translation techniques, known as RGB2TIR GAN, we address the scarcity of thermal image datasets, enhancing the model’s performance in diverse environments. We also present a post-processing method aimed at preserving edge features, thereby improving the dataset quality and model accuracy. This study demonstrates the first successful application of thermal images for panoptic segmentation, highlighting the potential for expanding the model’s applicability across various environmental conditions and suggesting avenues for future research in domain- specific models.