온라인 배너 광고 강화학습의 최적 탐색-활용 전략: 구전효과의 영향
Optimal Exploration-Exploitation Strategies in Reinforcement Learning for Online Banner Advertising: The Impact of Word-of-Mouth Effects
0
조회
0
다운로드
0
인용
0
참고문헌
키워드
초록
온라인 배너 광고 산업에서는 일반적으로 복수의 배너 대안이 제작된다. 이때 중요한 의사결정은 어떤 광고 배너 대안을선택해서고객에게노출하느냐하는것이다. 각배너대안을고객이클릭할확률을미리알수없기때문에경영자는실험적으로여러 대안을 노출한 후, 고객의 클릭 여부에 따라 각 대안의 클릭 확률을 추정하며 최적의 대안을 찾아야 하고 이것은온라인 광고와 관련된 강화학습 프로세스이다. 이 과정에서의 주요 의사결정 문제는 축적된 추정 클릭 확률 지식을 이용해서 최적의 대안을 노출하는 활용 전략과, 잠재적으로 더 우수한 대안을 찾기 위해 새로운 대안을 시도해보는 탐색 전략의 최적 균형점을 찾는 것이다. 본 연구는 구전효과와 대안의 수가 이러한 최적 탐색-활용 전략에 미치는 영향을 분석하였다. 이는 고객이 노출된 배너를 클릭하는경우 관련제품을 주위에 홍보하는 과정을 통해 광고배너의 클릭률이 높아지는 구전효과를 온라인 광고 관련 강화학습에 추가하여 구현한 것이다. 분석을 위해 Multi-Armed Bandit 모형을 이용한 시뮬레이션 기법을 사용하였다. 분석 결과, 구전효과의 크기가 커지고 배너 대안의 수가 적을수록 광고 강화학습의 최적 탐색 수준이 높아지는 것이 관측되었다. 이는 구전효과에 의해 고객이 광고 배너를 클릭할 확률이 증가함에 따라 기존에 축적했던 추정 클릭률 지식의 가치가 낮아지고, 따라서 새로운 대안을 탐색하는 것의 가치가 증가하기 때문으로 분석되었다. 또한 광고 대안의 수가 작을 경우에는 구전효과 크기가 커질 때 최적 탐색 수준이 더 큰 폭으로 증가하는 경향을 발견하였다. 최근 온라인 구전으로 인해 구전효과의 영향이 커지는 시점에서 본 연구는 의미 있는 시사점을 제공한다.
Abstract
One of the most important decisions for managers in the online banner advertising industry, is to choose the best banner alternative for exposure to customers. Since it is difficult to know the click probability of each banner alternative in advance, managers must experiment with multiple alternatives, estimate the click probability of each alternative based on customer clicks, and find the optimal alternative. In this reinforcement learning process, the main decision problem is to find the optimal balance between the level of exploitation strategy that utilizes the accumulated estimated click probability information and exploration strategy that tries new alternatives to find potentially better options. In this study we analyze the impact of word-of-mouth effects and the number of alternatives on the optimal exploration-exploitation strategies. More specifically, we focus on the word-of-mouth effect, where the click-through rate of the banner increases as customers promote the related product to those around them after clicking the exposed banner, and add it to the overall reinforcement learning process. We analyze our problem by employing the Multi-Armed Bandit model, and the analysis results show that the larger the word-of-mouth effect and the fewer the number of banner alternatives, the higher the optimal exploration level of advertising reinforcement learning. We find that as the probability of customers clicking on the banner increases due to the word-of-mouth effect, the value of the previously accumulated estimated click-through rate knowledge decreases, and therefore the value of exploring new alternatives increases. Additionally, when the number of advertising alternatives is small, a larger increase in the optimal exploration level was observed as the magnitude of the word-of-mouth effect increased. This study provides meaningful academic and managerial implications at a time when online word-of-mouth and its impact on society and business is becoming more important.
참고문헌 (43)
- "통계청" (2011)[URL]
- Auer, P., "Finite-time analysis of the multiarmed bandit problem", Machine Learning, 47(2), pp.235~256 (2002)
- Bakos, Y., "Cooperation without Enforcement? A Comparative Analysis of Litigation and Online Reputation as Quality Assurance Mechanisms", Management Science, 57(11), pp.1944~1962 (2011)
- Bone, P. F., "Word-of-Mouth Effects on Short-Term and Long-Term Product Judgements", Journal of Business Research, 32(3), pp.213~223 (1995)
- Burton, R.M., "Computational modeling for what-is, what-might-be, and what-should-be studies-and triangulation", Organization Science, 22(5), pp.1195~1202 (2011)
- Chandon, J. L., "Effects of configuration and exposure levels on responses to web advertisements", Journal of Advertising, 43(3), pp.217~229 (2003)
- Chu, W., "A case study of behavior-driven conjoint analysis on Yahoo! Front Page Today module", Proc. 15th ACM SIGKDD International Conference Knowledge Discovery Data Mining, pp.1097~1104 (2009)
- Daw, N. D., "Cortical substrates for exploratory decisions in humans", Nature, 441(7095), pp.876~879 (2006)
- Dichter, E., "How Word-of Mouth Advertising Works", Harvard Business Review, 44(6), pp.147~166 (1966)
- Duan, W., "The dynamics of online word-of-mouth and product sales - An empirical investigation of the movie industry", Journal of Retailing, 84(2), pp.233~242 (2008)
- Ferecatu, A., "Understanding managers' trade-offs between exploration and exploitation", Marketing Science, 41(1), pp.139~165 (2022)
- Floyd, K., "How Online Product Reviews Affect Retail Sales: A Meta-analysis", Journal of Retailing, 90(2), pp.217~232 (2014)
- Gu, b., "Research Note - The Impact of External Word-of-Mouth Sources on Retailer Sales of High-Involvement Products", Information Systems Research, 23(1), pp.182~196 (2012)
- Hanssens, D. M., "Handbook of Research on Customer Equity in Marketing", pp.178~198 (2015)
- Hauser, J. R., "Website morphing 2.0: Switching costs, partial exposure, random exit, and when to morph", Management Science, 60(6), pp.1594~1616 (2014)
- Iyer, G., "The targeting of advertising", Marketing Science, 24(3), pp.461~476 (2005)
- Li, H., "Cognitive impact of banner ad characteristics: An experimental study", Journalism and Mass Communication Quarterly, 76(2), pp.341~353 (1999)
- Liberali, G., "Morphing for consumer dynamics: Bandits meet hidden markov models", Marketing Science, 41(4), pp.769~794 (2022)
- Liu, Y., "Word of mouth for movies: Its dynamics and impact on box office revenue", Journal of Marketing, 70(3), pp.74~89 (2006)
- Lothia, R., "The impact of content and design elements on banner advertising click-through rates", Journal of Advertising Research, 43(4), pp.410~418 (2003)
- Luce, R., "Individual Choice Behavior: A Theoretical Analysis" (1959), Wiley
- Luger, J., "Dynamic balancing of exploration and exploitation: The contingent benefits of ambidexterity", Organization Science, 29(3), pp.449~470 (2018)
- March, J. G., "Exploration and exploitation in organizational learning", Organization Science, 2(1), pp.71~87 (1991)
- Mahajan, A., "Foundations and applications of sensor management", pp.121~151 (2008), Springer US
- Mnih, V., "Human-level control through deep reinforcement learning", Nature, 518(7540), pp.529~533 (2015)
- Namin, A., "Impact of message design on banner advertising involvement and effectiveness: An empirical investigation", Journal of Marketing Communications, 26(2), pp.115~129 (2020)
- North, M., "Click Me: An Examination of the Impact Size, Color, and Design has on Banner Advertisements Generating Clicks", Journal of Financial Services Marketing, 22, pp.99~108 (2017)
- Park, J.W., "A Study on Performance Improvement with Combination of Softmax and UCB Algorithm", Journal of The Korea Society of Information Technology Policy & Management, 10(1), pp.649~654 (2018)
- Posen, H.E., "Chasing a moving target: Exploitation and exploration in dynamic environments", Management Science, 58(3), pp.587~601 (2012)
- Radighieri, J. P., "The Impact of source effects and message valence on word of mouth retransmission", International Journal of Market Research, 56(2), pp.249~263 (2014)
- Robbins, H., "Some aspects of the sequential design of experiments", Bulletin of the AmericanMathematical Society, 58(5), pp.527~535 (1952)
- Robinson, H., "Internet Advertising Effectiveness: the Effect of Design on Click-through Rates for Banner Ads", International Journal of Advertising, 26(4), pp.527~541 (2007)
- Rojas-Cordova, C., "Why one strategy does not fit all: a systematic review on exploration-exploitation in different organizational archetypes", Review of Managerial Science, 17(7), pp.2251~2295 (2023)
- Schwartz, E. M., "Customer acquisition via display advertising using multi-armed bandit experiments", Marketing Science, 36(4), pp.500~522 (2017)
- Senecal, S., "The influence of online product recommendations on consumers' online choices", Journal of Retailing, 80(2), pp.159~169 (2004)
- A Heuristic Approach to Explore: The Value of Perfect Information, "Shahrokhi T.S.", Johns Hopkins Carey Business School, ~, pp.Johns Hopkins Carey Business School Research Paper, (19-05) (2019)
- Smith, W.K., "Managing strategic contradictions: A top management model for managing innovation streams", Organization Science, 16(5), pp.522~536 (2005)
- s any publicity good publicity? A Note on the Impact of Boor Reviews, "Sorensen, A.T.", NBER, Stanford University, ~ (2004)
- Stanaland, A. J. S., "The impact of surfer", seeker mode on the effectiveness of website characteristics, 2010, pp.29(4), 569~595
- Sutton, R.S., "Reinforcement learning: An introduction" (2018), MIT press
- Uotila, J., "Exploration, exploitation, and variability: Competition for primacy revisited", Strategic Organization, 15(4), pp.461~480 (2017)
- Urban, G. L., "Morphing banner advertising", Marketing Science, 33(1), pp.27~46 (2014)
- Zhang, Z., "The impact of e-word-of-mouth on the online popularity of restaurants: A comparison of consumer reviews and editor reviews", International Journal of Hospitality Management, 29(4), pp.694~700 (2010)