중국 소셜미디어 크롤링과 텍스트 분석: SNS 말뭉치 기반 연구방법론의 모색

Abstract

Recently, social media has emerged as a new space for information exchange and a leading channel in setting trends within China's digital environment. With the quantitative growth of social media platforms, the accumulation of users' opinions and evaluations has naturally led to the formation of a vast Chinese corpus, which continues to expand rapidly. Consequently, exploring social media texts to understand the embedded social and cultural meanings has become a crucial academic task. Theoretically, all data visible on a web browser can be collected through crawling. In light of this, this paper expands the scope of web crawling to include Chinese social media content, investigating the data crawling processes on XiaoHongShu (小红书), BiliBili (哔哩哔哩), and Douyin (抖音). This paper introduces the prominent Chinese social media platforms such as XiaoHongShu, BiliBili, and Douyin, highlighting their unique characteristics, influence, as well as the necessity of data collection. Subsequently, it step-by-step investigates the entire process of collecting social media content, from securing hidden URLs to accessing platforms through user authentication, and elaborates on two data collection methods based on GUI and CLI. Lastly, the paper presents a case study of SNS text mining by analyzing comments on Korean cosmetic surgery (韩国整容) contents posted on social media. This paper contributes to understanding Chinese social media and its content crawling methodologies, proposing a new approach for corpus-based research on SNS. The social media crawling techniques and content analysis presented in this study are expected to enable a deeper understanding of the rich linguistic and cultural features of Chinese social media content.