트랜스포머 기반의 다중 시점 3차원 인체자세추정
Multi-View 3D Human Pose Estimation Based on Transformer
2
조회
0
다운로드
0
인용
0
참고문헌
키워드
초록
3차원 인체자세추정은 스포츠, 동작인식, 영상매체의 특수효과 등의 분야에서 널리 활용되고 있는 기술이다. 이를 위한 여러 방법들 중 다중 시점 3차원 인체자세추정은 현실의 복잡한 환경에서도 정밀한 추정을 하기 위해 필수적인 방법이다. 하지만 기존 다중 시점 3차원 인체자세추정 모델들은 3차원 특징 맵을 사용함에 따라 시간 복잡도가 높은 단점이 있다. 본 논문은 계산 복잡도가 적은 트랜스포머 기반 기존 단안 시점 다중 프레임 모델을 다중 시점에 대한 3차원 인체자세추정으로 확장하는 방법을 제안한다. 다중 시점으로 확장하기 위하여 먼저 2차원 인체자세 검출자 CPN(Cascaded Pyramid Network)을 활용하여 획득한 4개 시점의 17가지 관절에 대한 2차원 관절좌표를 연결한 8차원 관절좌표를 생성한다. 그 다음 이들을 패치 임베딩 한 뒤 17×32 데이터로 변환하여 트랜스포머 모델에 입력한다. 마지막으로, 인체자세를 출력하는 MLP(Multi-Layer Perceptron) 블록을 매 반복 마다 사용한다. 이를 통해 4개 시점에 대한 3차원 인체자세추정을 동시에 수정한다. 입력 프레임 길이 27을 사용한 Zheng[5]의 방법과 비교했을 때 제안한 방법의 모델 매개변수의 수는 48.9%, MPJPE(Mean Per Joint Position Error)는 20.6mm(43.8%) 감소했으며, 학습 횟수 당 평균 학습 소요 시간은 20배 이상 빠르다.
Abstract
The technology of Three-dimensional human posture estimation is used in sports, motion recognition, and special effects of video media. Among various methods for this, multi-view 3D human pose estimation is essential for precise estimation even in complex real-world environments. But Existing models for multi-view 3D human posture estimation have the disadvantage of high order of time complexity as they use 3D feature maps. This paper proposes a method to extend an existing monocular viewpoint multi-frame model based on Transformer with lower time complexity to 3D human posture estimation for multi-viewpoints. To expand to multi-viewpoints our proposed method first generates an 8-dimensional joint coordinate that connects 2-dimensional joint coordinates for 17 joints at 4-vieiwpoints acquired using the 2-dimensional human posture detector, CPN(Cascaded Pyramid Network). This paper then converts them into 17×32 data with patch embedding, and enters the data into a transformer model, finally. Consequently, the MLP(Multi-Layer Perceptron) block that outputs the 3D-human posture simultaneously updates the 3D human posture estimation for 4-viewpoints at every iteration. Compared to Zheng[5]'s method the number of model parameters of the proposed method was 48.9%, MPJPE(Mean Per Joint Position Error) was reduced by 20.6 mm (43.8%) and the average learning time per epoch was more than 20 times faster.
참고문헌 (15)
- Julieta Martinez, "A simple yet effective baseline for 3d human pose estimation", Proceedings of the IEEE International Conference on Computer Vision, pp.2640~2649 (2017)
- Long Zhao, "Semantic graph convolutional networks for 3d human pose regression", Proceedings of the IEEE, 2019, 3425~3435
- Dario Pavllo, "3d human pose estimation in video with temporal convolutions and semi-supervised training", Proceedings of the IEEE, 2019, 7753~7762
- Ailing Zeng, "Learning Skeletal Graph Neural Networks for Hard 3D Pose Estimation", Proceedings of the IEEE, 2021, 11436~11445
- Ce Zheng, "3d human pose estimation with spatial and temporal transformers", Proceedings of the IEEE, 2021, 11656~11665
- Yihui He, "Epipolar transformers", Proceedings of the ieee, 2020, 7779~7788
- Karim Iskakov, "Learnable triangulation of human pose", Proceedings of the IEEE, 2019, 7718~7727
- Yilun Chen, "Cascaded pyramid network for multi-person pose estimation", Proceedings of the IEEE conference on computer vision and pattern recognition, 7103, pp.7103~7112 (2018)
- Alexey Dosovitskiy, "An image is worth 16x16 words: Transformers for image recognition at scale" (2020)
- Fuyang Huang, "Deepfuse: An imu-aware network for real-time 3d human pose estimation from multi-view image", Proceedings of the IEEE, 2020, 429~438
- Ashish Vaswani, "Attention is all you need", Advances in neural information processing systems, 30 (2017)
- Jingwei Xu, "Deep kinematics analysis for monocular 3d human pose estimation", Proceedings of the IEEE, 2020, 899~908
- Jingbo Wang, "Motion guided 3d pose estimation from videos", European Conference on Computer Vision, pp.764~780 (2020), Springer International Publishing
- Hui Shuai, "Adaptive multi-view and temporal fusing transformer for 3d human pose estimation", IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(4), pp.4122~4135 (2022)
- Catalin Ionescu, "Human3. 6m: Large scale datasets and predictive methods for 3d human sensing in natural environments", IEEE transactions on pattern analysis and machine intelligence, 36(7), pp.1325~1339 (2013)