Bidirectional Convolutional LSTM을 이용한 Deepfake 탐지 방법
A Method of Detection of Deepfake Using Bidirectional Convolutional LSTM
0
조회
0
다운로드
1
인용
0
참고문헌
키워드
초록
최근 하드웨어의 성능과 인공지능 기술이 발달함에 따라 육안으로 구분하기 어려운 정교한 가짜 동영상들이 증가하고 있다. 인공지능을 이용한 얼굴 합성 기술을 딥페이크라고 하며 약간의 프로그래밍 능력과 딥러닝 지식만 있다면 누구든지 딥페이크를 이용하여 정교한 가짜 동영상을 제작할 수 있다. 이에 무분별한 가짜 동영상이 크게 증가하였으며 이는 개인 정보 침해, 가짜 뉴스, 사기 등에 문제로 이어질 수 있다. 따라서 사람의 눈으로도 진위를 가릴 수없는 가짜 동영상을 탐지할 수 있는 방안이 필요하다. 이에 본 논문에서는 Bidirectional Convolutional LSTM과 어텐션 모듈(Attention module)을 적용한 딥페이크 탐지 모델을 제안한다. 본 논문에서 제안하는 모델은 어텐션 모듈과 신경곱 합성망 모델을 같이 사용되어 각 프레임의 특징을 추출하고 기존의 제안되어왔던 시간의 순방향만을 고려하는 LSTM과 달리 시간의 역방향도 고려하여 학습한다. 어텐션 모듈은 합성곱 신경망 모델과 같이 사용되어 각 프레임의 특징 추출에 이용한다. 실험을 통해 본 논문에서 제안하는 모델은 93.5%의 정확도를 갖고 기존 연구의 결과보다 AUC가 최대 50% 가량 높음을 보였다.
Abstract
With the recent development of hardware performance and artificial intelligence technology, sophisticated fake videos that are difficult to distinguish with the human’s eye are increasing. Face synthesis technology using artificial intelligence is called Deepfake, and anyone with a little programming skill and deep learning knowledge can produce sophisticated fake videos using Deepfake. A number of indiscriminate fake videos has been increased significantly, which may lead to problems such as privacy violations, fake news and fraud. Therefore, it is necessary to detect fake video clips that cannot be discriminated by a human eyes. Thus, in this paper, we propose a deep-fake detection model applied with Bidirectional Convolution LSTM and Attention Module. Unlike LSTM, which considers only the forward sequential procedure, the model proposed in this paper uses the reverse order procedure. The Attention Module is used with a Convolutional neural network model to use the characteristics of each frame for extraction. Experiments have shown that the model proposed has 93.5%accuracy and AUC is up to 50% higher than the results of pre-existing studies.
참고문헌 (42)
- B. Dolhansky, "The dee pfake detection challenge (DFDC) pre view dataset", arXiv preprint arXiv:1910.08854, ~ (2019)
- K. Li, "Tell me where to look: Gui ded attention inference network", arXi v preprint arXiv:1802.10171, ~ (2018)
- E. Sabir, "Recurrent Convolutional Strategies for Face Manipulation Detection in Vi deos", Proc. IEEE, 2019, ~
- W. S. Hu, "Feature extraction a nd classification based on spatial-spec tral convlstm neural network for hype rspectral images", arXiv preprint arXi v:1905.03577, ~ (2019)
- Y. Li, "Exposing deepfake videos by detecting face warping artif acts", arXiv preprint arXiv:1811.00656, ~ (2018)
- P. Korshunov, "Deepfak es: a new threat to face recognition? assessment and detection", arXiv prep rint arXiv:1812.08685, ~ (2018)
- T.T. Nguyen, "Deep Learning fo r Deepfakes Creation and Detection", arXiv preprints, arXiv:1909.11573, ~ (2019)
- Q. Liu, "Bidirectional-convolutional LSTM based spectral-spatial feature learnin g for hyperspectral image classification", Remote Sens, 9(12), pp.1330~ (2017)
- J. Deng, "Ar cFace: Additive angular margin loss f or deep face recognition", arXiv:1801.07698, ~ (2018)
- H. Fukui, "Attention branch network: Learning of attention mechanism for visual explanation", Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2019)
- A. Hanson, "Bidirectional convolutio nal LSTM for the detection of violenc e in videos", LNCS, 11130, pp.280~295 (2019)
- S. Woo, "Cbam: Convolutional block at tention module", Proceedings of the European Conference on Computer Vi sion (ECCV), pp.3~19 (2018)
- Y. Li, "Cel eb-DF: A Large-scale Challenging Dat aset for DeepFake Forensics", IEEE Conf. on Computer Vision and Patten Recognition (CVPR) (2020)
- S. Xingjian, "Convolu tional LSTM network: A machine lear ning approach for precipitation nowca sting", Neural Information Processing Systems (2015)
- O. M. Parkhi, "Deep face recognition", Proceedings of the British Machine Vision, 1(3), pp.6~ (2015)
- I. Amerini, "Deepfake Video Detectio n through Optical Flow based CNN", Proc. IEEE, 2019, ~
- D. Guera, "Deepfake vid eo detection using recurrent neural n etworks", AVSS (2018)
- M. Tan, "EfficientNet: R ethinking model scaling for convolutio nal neural networks", International Conference on Machine Learning (2019)
- D. A. Pitaloka, "Enhancing cnn with preprocessing stage in auto matic emotion recognition", Procedia Computer Science, 116, pp.523~529 (2017)
- A. Rossler, "Faceforensics++: Learning to detect manipulated facial images", arXiv pre print arXiv:1901.08971, ~ (2019)
- H. Dang, "On the Detection of Digital Face Manipulation", Proc. I EEE, 2020, ~
- C. Ding, "Robust face reco gnition via multimodal deep face repr esentation", IEEE TMM, 17(11), pp.2049~2058 (2015)
- L. Chen, "SCACNN: Spatial and channel-wise attent ion in convolutional networks for ima ge captioning", CVPR (2017)
- S. Suwajanakorn, "Synthesizing Obama: learning lip sync from audio", ACM Transactions on Graphics (T OG), 36(4) (2017)
- Y. Taigman, "Deepface: Closing the ga p to human-level performance in face verification", CVPR (2014)
- R. Tolosana, "DeepFakes and beyond: A survey of face manipulation and fake detection", Inf. Fusion (2020)
- A. Singh, "DeepFake Video Detecti on: A Time-Distributed Approach", S N Computer Science (2020)
- K. Dale, "Video face replacement", Proceedings o f the 2011 SIGGRAPH Asia conference, pp.1~10 (2011)
- J. Thies, "Face2Face: Real-time face capture an d reenactment of RGB videos", CVPR (2016)
- F. Chollet, "Xception: Deep learning with depthwise separable convolutions", Proceedings of the IEEE confere nce on computer vision and pattern recognition, pp.1251~1258 (2017)
- C. Szegedy, "Rethinking the inception architecture for computer vision", Proceedings of the IEEE conference on computer vision and pattern recognition, pp.2818~2826 (2016)
- J. Hu, "Squeeze-and -excitation networks", Proceedings of the IEEE conference on computer v ision and pattern recognition, pp.7132~7141 (2018)
- K. He, "Deepresidual learning for image recognition", Proceedings of the IEEE conference on computer vision and pattern recognition, pp.770~778 (2016)
- DP. Kingma, "Auto-enc oding variational bayes", ICLR (2014)
- D.J. Rezende, "Stochastic backpropagation an d approximate inference in deep gener ative models", ICML (2014)
- I. Goodfellow, "Genera tive adversarial nets", NeurIPS (2014)
- "ZAO"[URL]
- "FakeApp"[URL]
- "The state of DeepFakes: landsc ape, threats, and impact" (2019)[URL]
- 김민서, "STFT와 RNN을 활용한 화자 인증 모델", 정보보호학회논문지, 29(6), pp.1393~1401 (2019), 한국정보보호학회[DOI]
- S. Bianco, "Benchmark analysis of representative deep neural network ar chitectures", IEEE Access, 6 (2018)
- V.J. Reddi, "Mlperf inference be nchmark", arXiv preprint arXiv:1911.02549, ~ (2019)