Image Understanding for Visual Dialog
0
조회
0
다운로드
1
인용
0
참고문헌
키워드
초록
This study proposes a deep neural network model based on an encoder–decoder structure for visual dialogs. Ongoing linguistic understanding of the dialog history and context is important to generate correct answers toquestions in visual dialogs followed by questions and answers regarding images. Nevertheless, in many cases, avisual understanding that can identify scenes or object attributes contained in images is beneficial. Hence, inthe proposed model, by employing a separate person detector and an attribute recognizer in addition to visualfeatures extracted from the entire input image at the encoding stage using a convolutional neural network, weemphasize attributes, such as gender, age, and dress concept of the people in the corresponding image and usethem to generate answers. The results of the experiments conducted using VisDial v0.9, a large benchmarkdataset, confirmed that the proposed model performed well.
Abstract
This study proposes a deep neural network model based on an encoder–decoder structure for visual dialogs. Ongoing linguistic understanding of the dialog history and context is important to generate correct answers toquestions in visual dialogs followed by questions and answers regarding images. Nevertheless, in many cases, avisual understanding that can identify scenes or object attributes contained in images is beneficial. Hence, inthe proposed model, by employing a separate person detector and an attribute recognizer in addition to visualfeatures extracted from the entire input image at the encoding stage using a convolutional neural network, weemphasize attributes, such as gender, age, and dress concept of the people in the corresponding image and usethem to generate answers. The results of the experiments conducted using VisDial v0.9, a large benchmarkdataset, confirmed that the proposed model performed well.
참고문헌 (9)
- A. Agrawal, "VQA : visual question answering", International Journal of Computer Vision, 123(1), pp.4~31 (2017)
- A. Das, "Visual dialog", Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp.326~335 (2017)
- P. H. Seo, "Visual reference resolution using attention memory for visual dialog", Advances in Neural Information Processing Systems, 30, pp.3719~3729 (2017)
- J. Lu, "Best of both worlds : transferring knowledge from discriminative learning to a generative visual dialog model", Advances in Neural Information Processing Systems, 30, pp.314~324 (2017)
- Q. Wu, "Are you talking to me? Reasoned visual dialog generation through adversarial learning", Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp.6106~6115 (2018)
- P. Anderson, "Bottom-up and top-down attention for image captioning and visual question answering", Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp.6077~6086 (2018)
- L. Peng, "Word-to-region attention network for visual question answering", Multimedia Tools and Applications, 78(3), pp.3843~3858 (2019)
- A. Trott, "Interpretable counting for visual question answering", Proceedings of the 6th International Conference on Learning Representations (2018)
- M. T. Desta, "Object-based reasoning in VQA", Proceedings of 2018 IEEE Winter Conference on Applications of Computer Vision (WACV), pp.1814~1823 (2018)