효과적인 인간-로봇 상호작용을 위한 딥러닝 기반 로봇 비전 자연어 설명문 생성 및 발화 기술
Robot Vision to Audio Description Based on Deep Learning for Effective Human-Robot Interaction
0
조회
0
다운로드
2
인용
0
참고문헌
키워드
초록
For effective human-robot interaction, robots need to understand the current situation context well, but also the robots need to transfer its understanding to the human participant in efficient way. The most convenient way to deliver robot’s understanding to the human participant is that the robot expresses its understanding using voice and natural language. Recently, the artificial intelligence for video understanding and natural language process has been developed very rapidly especially based on deep learning. Thus, this paper proposes robot vision to audio description method using deep learning. The applied deep learning model is a pipeline of two deep learning models for generating natural language sentence from robot vision and generating voice from the generated natural language sentence. Also, we conduct the real robot experiment to show the effectiveness of our method in human-robot interaction.
Abstract
For effective human-robot interaction, robots need to understand the current situation context well, but also the robots need to transfer its understanding to the human participant in efficient way. The most convenient way to deliver robot’s understanding to the human participant is that the robot expresses its understanding using voice and natural language. Recently, the artificial intelligence for video understanding and natural language process has been developed very rapidly especially based on deep learning. Thus, this paper proposes robot vision to audio description method using deep learning. The applied deep learning model is a pipeline of two deep learning models for generating natural language sentence from robot vision and generating voice from the generated natural language sentence. Also, we conduct the real robot experiment to show the effectiveness of our method in human-robot interaction.
참고문헌 (35)
- J. Donahue, "Long-term recurrent convolutional networks for visual recognition and description", 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2015)
- O. Vinyals, "Show and tell: A neural image caption generator", 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2015)
- L. Yao, "Describing videos by exploiting temporal structure", 2015 IEEE International Conference on Computer Vision (ICCV) (2015)
- O. Russakovsky, "ImageNet Large Scale Visual Recognition Challenge", ~ (2014)
- K. Simonyan, "Very deep convolutional networks for large-scale image recognition", ~ (2014)
- H. Yu, "Video Paragraph Captioning Using Hierarchical Recurrent Neural Networks", ~ (2016)
- L. Gao, "Video captioning with attention-based LSTM and semantic consistency", IEEE Transactions on Multimedia, 9(9), pp.2045~2055 (2017)
- K. Tokuda, "An HMM-based speech synthesis system applied to English", 2002 IEEE Workshop on Speech Synthesis (2002)
- A. van den Oord, "WaveNet : A generative model for raw audio", ~ (2016)
- J. Sotelo, "Char2Wav : End-to-end speech synthesis", ICLR 2017 (2017)
- S. O. Arik, "Deep voice : Real-time neural text-to-speech", ~ (2017)
- O. Vinyals, "Grammar as a foreign language", ~ (2015)
- M. -T. Luong, "Effective approaches to attention-based neural machine translation", ~ (2015)
- S. Venugopalan, "Sequence to Sequence -- Video to Text", 2015 IEEE International Conference on Computer Vision (ICCV) (2015)
- K. Cho, "On the properties of neural machine translation: Encoderdecoder approaches", ~ (2014)
- I. Sutskever, "Sequence to sequence learning with neural networks", ~ (2014)
- S. Hochreiter, "Long short-term memory", Neural Computation, 9(8) (1997)
- D. P. Kingma, "Adam : A method for stochastic optimization", ~ (2014)
- Y. Wang, "Tacotron : Towards End-to-End Speech Synthesis", Interspeech 2017 (2017)
- D. Bahdanau, "Neural machine translation by jointly learning to align and translate", ~ (2014)
- N. Srivastava, "Dropout : A simple way to prevent neural networks from overfitting", Journal of Machine Learning Research, 15(1), pp.1929~1958 (2014)
- J. Lee, "Fully character-level neural machine translation without explicit segmentation", Transactions of the Association for Computational Linguistics, 5, pp.365~378 (2017)
- R. K. Srivastava, "Highway networks", ~ (2015)
- J. Chung, "Empirical evaluation of gated recurrent neural networks on sequence modeling", ~ (2014)
- S. Ioffe, "Batch normalization : Accelerating deep network training by reducing internal covariate shift", ~ (2015)
- K. He, "Deep residual learning for image recognition", 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp.770~778 (2016)
- Y. Wu, "Google’s neural machine translation system: Bridging the gap between human and machine translation", ~ (2016)
- H. Zen, "Fast, compact, and high-quality LSTM-RNN based statistical parametric speech synthesizers for mobile devices", Interspeech (2016)[DOI]
- D. Griffin, "Signal estimation from modified short-time fourier transform", ICASSP '83. IEEE International Conference on Acoustics, Speech, and Signal Processing, 32(2), pp.236~243 (1984)
- M. Abadi, "TensorFlow:Large-scale machine learning on heterogeneous distributed systems", ~ (2016)
- D. L. Chen, "Collecting highly parallel data for paraphrase evaluation", 49th Annual Meeting of the Association for Computational Linguistics, 190~200 (2011)
- K. Papineni, "Bleu : a method for automatic evaluation of machine translation", 40th Annual Meeting on Association for Computational Linguistics, pp.311~318 (2002)
- C.-Y. Lin, "ROUGE: A Package for Automatic Evaluation of Summaries", ACL-04 Workshop, pp.74~81 (2004)
- M. Denkowski, "Meteor universal: Language specific translation evaluation for any target language", Ninth Workshop on Statistical Machine Translation, pp.376~380 (2014)
- R. Vedantam, "CIDEr: Consensus- based image description evaluation", 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp.4566~4575 (2015)