Multimodal Context Embedding for Scene Graph Generation
2
조회
0
다운로드
1
인용
0
참고문헌
키워드
초록
This study proposes a novel deep neural network model that can accurately detect objects and their relationshipsin an image and represent them as a scene graph. The proposed model utilizes several multimodal features,including linguistic features and visual context features, to accurately detect objects and relationships. Inaddition, in the proposed model, context features are embedded using graph neural networks to depict thedependencies between two related objects in the context feature vector. This study demonstrates theeffectiveness of the proposed model through comparative experiments using the Visual Genome benchmarkdataset.
Abstract
This study proposes a novel deep neural network model that can accurately detect objects and their relationshipsin an image and represent them as a scene graph. The proposed model utilizes several multimodal features,including linguistic features and visual context features, to accurately detect objects and relationships. Inaddition, in the proposed model, context features are embedded using graph neural networks to depict thedependencies between two related objects in the context feature vector. This study demonstrates theeffectiveness of the proposed model through comparative experiments using the Visual Genome benchmarkdataset.
참고문헌 (10)
- Y. Li, "Scene graph generation from objects, phrases and region captions", Proceedings of the IEEE International Conference on Computer Vision, pp.1261~1270 (2017)
- J. Yang, "Graph R-CNN for scene graph generation", Proceedings of the 15th European Conference on Computer Vision(ECCV), pp.670~685 (2018)
- W. Liao, "Natural language guided visual relationship detection", Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp.444~453 (2019)
- R. Krishna, "Visual Genome : connecting language and vision using crowdsourced dense image annotations", International Journal Of Computer Vision, 123(1), pp.32~73 (2017)
- B. Dai, "Detecting visual relationships with deep relational networks", Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp.3076~3086 (2017)
- N. Gkanatsios, "Deeply supervised multimodal attentional translation embeddings for visual relationship detection", Proceedings of 2019 IEEE International Conference on Image Processing, pp.1840~1844 (2019)
- J. Gu, "Scene graph generation with external knowledge and image reconstruction", Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp.1969~1978 (2019)
- M. Qi, "Attentive relational networks for mapping images to scene graphs", Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp.3957~3966 (2019)
- S. Woo, "LinkNet : relational embedding for scene graph", Advances in Neural Information Processing Systems, 31, pp.560~570 (2018)
- T. N. Kipf, "Semi-supervised classification with graph convolutional networks", Proceedings of the 5th International Conference on Learning Representations (ICLR) (2017)