딥페이크 탐지 모델의 검증 방법론 불일치에 따른 성능 편향 분석 연구
On the Performance Biases Arising from Inconsistencies in Evaluation Methodologies of Deepfake Detection Models
0
조회
0
다운로드
0
인용
0
참고문헌
키워드
초록
생성형 AI 기술이 정교해짐에 따라 빈번해지는 악의적 딥페이크 사용에 대응하기 위해 딥페이크 탐지 모델 연구가 활발히 진행되고 있다. 딥페이크 탐지 모델의 성능 평가는 학습 데이터셋 선택, 데이터셋 전처리, 학습 방법, 평가 데이터셋 선택 과정을 순차적으로 거친다. 하지만 기존 딥페이크 탐지 연구들은 각 단계마다 임의로 검증 방법론을 선택하여 논문에서의 성능이 표준화된 환경에서는 재현되지 않는 성능 평향 문제가 발생한다. 본 논문에서는 기존 딥페이크 탐지 연구의 검증 방법론을 분석하여 성능 평가의 신뢰성 저하 원인을 파악한다. 나아가 표준화된 환경에서의 실험을 통해 탐지 모델의 절대적 성능 비교에 어려움이 있음을 보여준다. 본 연구의 실험 결과는 탐지 성능 평가 신뢰성 제고와 절대적 성능 비교를 위해서는 통일된 검증 방법론이 필요함을 제시한다.
Abstract
As deepfake technology advances, its increasing misuse has spurred extensive research into detection models. These models’ performance evaluations, which include selecting train and test datasets, data preprocessing, and data augmentation, are often compromised by arbitrarily chosen validation methodologies in existing studies. This leads to biases under standardized conditions. This paper reviews these methodologies to pinpoint what diminishes evaluation reliability. Experiments in standardized environments reveal the difficulties in comparing performance absolutely. The findings highlighted the need for a consistent validation methodology to boost evaluation reliability and enable fair comparisons.
참고문헌 (20)
- L. Li, "Face x-ray for more general face forgery detection", Proceedings of the IEEE, 2020, 5001~5010
- L. Chen, "Self-supervised learning of adversarial example: Towards good generalizations for deepfake detection", Proceedings of the IEEE, 2022, 18710~18719
- H. Liu, "Spatial-phase shallow learning: rethinking face forgery detection in frequency domain", Proceedings of the IEEE, 2021, 772~781
- Y. Luo, "Generalizing face forgery detection with highfrequency features", Proceedings of the IEEE, 2021, 16317~16326
- Y. Qian, "European conference on computer vision", pp.86~103 (2020), Springer International Publishing
- A. Rossler, "Faceforensics++: Learning to detect manipulated facial images", Proceedings of the IEEE, 2019, 1~11
- Y. Li, "Celeb-df: A large-scale challenging dataset for deepfake forensics", Proceedings of the IEEE, 2020, 3207~3216
- L. Jiang, "Deeperforensics-1.0: A largescale dataset for real-world face forgery detection", Proceedings of the IEEE, 2020, 2889~2898
- J. Choi, "Exploiting Style Latent Flows for Generalizing Deepfake Video Detection", Proceedings of the IEEE, 2024, 1133~1143
- H. Nguyen, "Capsule-forensics: Using Capsule Networks to Detect Forged Images and Videos", IEEE international conference on acoustics, speech and signal processing, pp.2307~2311 (2019)
- H. Dang, "On the Detection of Digital Face Manipulation", Proceedings of the IEEE, 2020, 5781~5790
- B. Zi, "Wilddeepfake: A challenging real-world dataset for deepfake detection", Proceedings of the 28th ACM international conference on multimedia, pp.2382~2390 (2020)
- L. Li, "Advancing high fidelity identity swapping for forgery detection", Proceedings of the IEEE, 2020, 5074~5083
- Z. Wang, "Altfreezing for more general video face forgery detection", Proceedings of the IEEE, 2023, 4129~4138
- L. Chen, "Ost: Improving generalization of deepfake detection via one-shot test-time training", Advances in Neural Information Processing Systems, 35, pp.24597~24610 (2022)
- W. Zhuang, "IA-ViT: Unsupervised inconsis tency-aware method based on vision transformer for face forgery detection", European conference on computer vision, pp.391~407 (2022), Springer Nature Switzerland
- L. Chen, "Self-supervised learning of adversarial example: Towards good generalizations for deepfake detection", Proceedings of the IEEE, 2022, 18710~18719
- Z. Yan, "Ucf: Uncovering common features for generalizable deepfake detection", Proceedings of the IEEE, 2024, 22412~22423