GBGNN: Gradient Boosted Graph Neural Networks

Abstract

In recent years, graph neural networks (GNNs) have been extensively used to analyze graph data across variousdomains because of their powerful capabilities in learning complex graph-structured data. However, recentresearch has focused on improving the performance of a single GNN with only two or three layers. This isbecause stacking layers deeply causes the over-smoothing problem of GNNs, which degrades the performanceof GNNs significantly. On the other hand, ensemble methods combine individual weak models to obtain bettergeneralization performance. Among them, gradient boosting is a powerful supervised learning algorithm thatadds new weak models in the direction of reducing the errors of the previously created weak models. Afterrepeating this process, gradient boosting combines the weak models to produce a strong model with betterperformance. Until now, most studies on GNNs have focused on improving the performance of a single GNN. In contrast, improving the performance of GNNs using multiple GNNs has not been studied much yet. In thispaper, we propose gradient boosted graph neural networks (GBGNN) that combine multiple shallow GNNswith gradient boosting. We use shallow GNNs as weak models and create new weak models using the proposedgradient boosting-based loss function. Our empirical evaluations on three real-world datasets demonstrate thatGBGNN performs much better than a single GNN. Specifically, in our experiments using graph convolutionalnetwork (GCN) and graph attention network (GAT) as weak models on the Cora dataset, GBGNN achievesperformance improvements of 12.3%p and 6.1%p in node classification accuracy compared to a single GCNand a single GAT, respectively.