본문 바로가기
KERD
논문KCI 등재

Investigation on User Experience of Intelligent Personal Assistants from Online Reviews

2

조회

0

다운로드

0

인용

0

참고문헌

키워드

초록

In this paper, User Experience (UX) of Intelligent Personal Assistants (IPAs) was investigated through semantic network analysis. As a case study, UX of smart speakers which are a kind of embodied IPAs was investigated through semantic network analysis based on online reviews, identifying what factors are important to UX of IPAs for users. Although there were some limitations, this study contributed to understanding of UX of IPAs, by proposing eleven important factors for UX of IPAs and validating them. This study showed that most of the users satisfied the use of IPAs, although they felt that performance was not completely reliable, which was similar with previous studies related to IPAs. In addition, this study showed the aesthetic aspects are also important for users’ enjoyment, which was the interest result. The results of this study are noteworthy and considered to be reflected in the design of IPAs to improve UX of IPAs and satisfy users.

Abstract

In this paper, User Experience (UX) of Intelligent Personal Assistants (IPAs) was investigated through semantic network analysis. As a case study, UX of smart speakers which are a kind of embodied IPAs was investigated through semantic network analysis based on online reviews, identifying what factors are important to UX of IPAs for users. Although there were some limitations, this study contributed to understanding of UX of IPAs, by proposing eleven important factors for UX of IPAs and validating them. This study showed that most of the users satisfied the use of IPAs, although they felt that performance was not completely reliable, which was similar with previous studies related to IPAs. In addition, this study showed the aesthetic aspects are also important for users’ enjoyment, which was the interest result. The results of this study are noteworthy and considered to be reflected in the design of IPAs to improve UX of IPAs and satisfy users.

참고문헌 (84)

  1. Abdolrahmani, A., "Siri Talks at You: An Empirical Investigation of Voice-Activated Personal Assistant (VAPA) Usage by Individuals Who Are Blind", Proceedings of the 20th International ACM SIGACCESS Conference on Computers and Accessibility, pp.249~258 (2018)
  2. Aguirre-Munizaga, M., "An Embedded Prototype System for People with Disabilities Using Google’s Speech", International Conference Europe Middle East &North Africa Information Systems and Technologies to Support Learning, pp.301~311 (2018)
  3. John R. Anderson, "A propositional theory of recognition memory", Memory & Cognition, 2(3), pp.406~412 (1974), Springer Science and Business Media LLC[DOI]
  4. Arhippainen, L., "Empirical evaluation of user experience in two adaptive mobile application prototypes", MUM 2003. Proceedings of the 2nd International Conference on Mobile and Ubiquitous Multimedia, pp.27~34 (2003)
  5. Assad, M., "PersonisAD:Distributed, Active, Scrutable Model Framework for Context-Aware Services", International Conference on Pervasive Computing, pp.55~72 (2007)
  6. Output to Computers, "Human Factors Issues in the Application of Speech Input", pp.2002, Baber, C.
  7. Balasuriya, S. S., "Use of voice activated interfaces by people with intellectual disability", Proceedings of the 30th Australian Conference on Computer-Human Interaction - OzCHI ’18, pp.102~112 (2018)
  8. Ballard, B., "Designing the mobile user experience" (2007), John Wiley & Sons
  9. Balota, D. A., "Are lexical decisions a good measure of lexical access? The role of word frequency in the neglected decision stage", Journal of Experimental Psychology: Human Perception and Performance, 10(3), pp.340~ (1984)
  10. Balota, D. A., "The locus of word-frequency effects in the pronunciation task: Lexical access and", or production?, 1985, pp.24(1), 89~106
  11. Bargas-Avila, J. A., "Old wine in new bottles or novel challenges: a critical analysis of empirical studies of user experience", Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp.2689~2698 (2011)
  12. Rajeev Batra, "Measuring the hedonic and utilitarian sources of consumer attitudes", Marketing Letters, 2(2), pp.159~170 (1991), Springer Science and Business Media LLC[DOI]
  13. Beneteau, E., "Communication Breakdowns Between Families and Alexa", CHI, 243 (2019)[DOI]
  14. Botha, A., "Mobile user experience for voice services: A theoretical framework", ~ (2012)
  15. Branham, S. M., "Reading between the guidelines:How commercial voice assistant guidelines hinder accessibility for blind users", The 21st International ACM SIGACCESS Conference on Computers and Accessibility - ASSETS ’19, pp.446~458 (2019)
  16. Cho, J., "Mental models and home virtual assistants (HVAs)", Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems (2018)
  17. 나건, "Study on Design Research using Semantic Network Analysis", 대한인간공학회지, 34(6), pp.563~581 (2015), 대한인간공학회[DOI]
  18. Cowan, B., "What can i help you with?: infrequent users’ experiences of intelligent personal assistants", Proceedings of the 19th International Conference on Human-Computer Interaction with Mobile Devices and Services, 43 (2017)
  19. Davis, F. D., "A technology acceptance model for empirically testing new end-user information systems: Theory and results", pp.1985
  20. Fred D. Davis, "User Acceptance of Computer Technology: A Comparison of Two Theoretical Models", Management Science, 35(8), pp.982~1003 (1989), Institute for Operations Research and the Management Sciences (INFORMS)[DOI]
  21. Doerfel, M. L., "What constitutes semantic network analysis? A comparison of research and methodologies", Connections, 21(2), pp.16~26 (1998)
  22. Xing Fang, "Sentiment analysis using product review data", Journal of Big Data, 2(1), pp.5~ (2015), Springer Science and Business Media LLC[DOI]
  23. G Elera, R., "Interacting with Intelligent Assistants to Predict Consumer Satisfaction", ~ (2018)
  24. Lingling Gao, "A unified perspective on the factors influencing consumer acceptance of internet of things technology", Asia Pacific Journal of Marketing and Logistics, 26(2), pp.211~231 (2014), Emerald[DOI]
  25. Gao, Y., "Alexa, My Love: Analyzing Reviews of Amazon Echo", 2018 IEEE SmartWorld, Ubiquitous Intelligence & Computing, Advanced & Trusted Computing, Scalable Computing &Communications, Cloud & Big Data Computing, Internet of People and Smart City Innovation (SmartWorld, UIC, pp.ATC, CBDCom[DOI]
  26. Gartner, N, "Gartner says worldwide spending on VPA-enabled wireless speakers will top $2 billion by 2020", Gartner Newsroom, ~ (2020)
  27. Gregório, F., "Ubiquitous and context-aware computing modelling: Study of devices integration in their environment", International Conference on Intelligent Human Systems Integration, pp.423~428 (2019)
  28. Sangyeal Han, "Understanding adoption of intelligent personal assistants", Industrial Management & Data Systems, 118(3), pp.618~636 (2018), Emerald[DOI]
  29. Hassenzahl, M., "Needs, affect, and interactive products–Facets of user experience", Interacting with Computers, 22(5), pp.353~362 (2010)
  30. Hassenzahl, M., "How motivational orientation influences the evaluation and choice of hedonic and pragmatic interactive products: The role of regulatory focus", Interacting with Computers, 20(4–5), pp.473~479 (2008)
  31. Marc Hassenzahl, "The Effect of Perceived Hedonic Quality on Product Appealingness", International Journal of Human-Computer Interaction, 13(4), pp.481~499 (2001), Informa UK Limited[DOI]
  32. Marc Hassenzahl, "User experience - a research agenda", Behaviour & Information Technology, 25(2), pp.91~97 (2006), Informa UK Limited[DOI]
  33. Marcel Heerink, "Assessing Acceptance of Assistive Social Agent Technology by Older Adults: the Almere Model", International Journal of Social Robotics, 2(4), pp.361~375 (2010), Springer Science and Business Media LLC[DOI]
  34. Matthew B. Hoy, "Alexa, Siri, Cortana, and More: An Introduction to Voice Assistants", Medical Reference Services Quarterly, 37(1), pp.81~88 (2018), Informa UK Limited[DOI]
  35. 황해정, "빅데이터 분석을 활용한 사용자 경험 평가 방법론 탐색: 아마존 에코에 대한 온라인 리뷰 분석을 중심으로", 한국콘텐츠학회 논문지, 16(8), pp.517~528 (2016), 한국콘텐츠학회[DOI]
  36. ISO, "9241-210: 2008. Ergonomics of human system interaction-Part 210:Human-centred design for interactive systems (formerly known as 13407)", The International Organization for Standardization (ISO), ~ (2008)
  37. Xiang Ji, "Development and Application of a Classification System for Voice Intelligent Agents", International Journal of Human–Computer Interaction, 35(9), pp.787~795 (2018), Informa UK Limited[DOI]
  38. Kaasinen, E., "A user-centric view of intelligent environments : User expectations, user experience and user role in building intelligent environments", Computers, 2(1), pp.1~33 (2013)
  39. Kepuska, V., "Next-generation of virtual personal assistants (Microsoft Cortana, Apple Siri, Amazon Alexa and Google Home)", 2018IEEE 8th Annual Computing and Communication Workshop and Conference,CCWC 2018 (2018)
  40. Ketola, P., "Integrating usability with concurrent engineering in mobile phone development" (2002), Tampere University Press
  41. Khan, S., "GesCAD: an intuitive interface for conceptual architectural design", Proceedings of the 29th Australian Conference on Computer-Human Interaction, pp.402~406 (2017)
  42. Changsu Kim, "Effects of personal motivation and computing characteristics on ubiquitous mobile device usages", International Journal of Mobile Communications, 17(2), pp.127~ (2019), Inderscience Publishers[DOI]
  43. Kim, G. -W., "First Impression as a Dynamic Experience: An InterviewBased Study of Impression Formation and Change", pp.2016
  44. Ga-Won Kim, "Analysis of Consumer Value Using Semantic Network: The Comparison of Hierarchical and Nonhierarchical Value Structures", Human Factors and Ergonomics in Manufacturing & Service Industries, 26(3), pp.393~407 (2016), Wiley[DOI]
  45. Kiseleva, J., "Predicting user satisfaction with intelligent assistants", Proceedings of the 39th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp.45~54 (2016)
  46. Knote, R., "Classifying Smart Personal Assistants: An Empirical Cluster Analysis", Proceedings of the 52nd Hawaii International Conference on System Sciences, pp.2024~2033 (2019)
  47. Kocaballi, A. B., "Understanding and Measuring User Experience in Conversational Interfaces", Interacting with Computers, 31(2), pp.192~207 (2019)
  48. Koskela, T., "Evolution towards smart home environments: empirical evaluation of three user interfaces", Personal and Ubiquitous Computing, 8(3–4), pp.234~240 (2004)
  49. Kouroupetroglou, G., "Usability methodologies for real-life voice user interfaces", Journal of Information Technology and Web Engineering(IJITWE), 4(4), pp.78~94 (2009)
  50. Kowalski, J., "Older Adults and Voice Interaction: A Pilot Study with Google Home", ArXiv Preprint ArXiv:1903.07195, ~ (2019)
  51. Chaiwoo Lee, "User Acceptance of Voice Interfaces in the Automobile", Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 59(1), pp.1641~1645 (2015), SAGE Publications[DOI]
  52. 이정명, "의미연결망 분석을 통한 디스플레이형 인공지능 스피커의 사용자 경험 요인 연구 : 아마존 에코의 온라인 리뷰 분석을 중심으로", 문화기술의 융합, 5(3), pp.9~23 (2019), 국제문화기술진흥원[DOI]
  53. Lewis, J. R., "Standardized questionnaires for voice interaction design", Voice Interaction Design, 1(1) (2016)
  54. Lim, J. H., "Applying Semantic Network Analysis to Develop User Experience Assessment Model for Smart TV", Applied Science, 9(16), pp.3307~ (2019)
  55. Irene Lopatovska, "User recommendations for intelligent personal assistants", Journal of Librarianship and Information Science, 52(2), pp.577~591 (2020), SAGE Publications[DOI]
  56. López, G., "Alexa vs. Siri vs. Cortana vs. Google Assistant: A Comparison of Speech-Based Natural User Interfaces", International Conference on Applied Human Factors and Ergonomics, pp.241~250 (2018)
  57. Meyer, D., "Text mining infrastructure in R", Journal of Statistical Software, 25(5), pp.1~54 (2008)
  58. Myers, C., "Patterns for How Users Overcome Obstacles in Voice User Interfaces", Proceedings of the 2018CHI Conference on Human Factors in Computing Systems - CHI ’18 (2018)
  59. Myers, C. M., "The Impact of User Characteristics and Preferences on Performance with an Unfamiliar Voice User Interface", Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, 47 (2019)
  60. Oded Netzer, "Mine Your Own Business: Market-Structure Surveillance Through Text Mining", Marketing Science, 31(3), pp.521~543 (2012), Institute for Operations Research and the Management Sciences (INFORMS)[DOI]
  61. Nowacki, C., "Improving the Usability of Voice User Interfaces: A New Set of Ergonomic Criteria" (2020)
  62. Orehovački, T., "Modelling the perceived pragmatic and hedonic quality of intelligent personal assistants", Advances in Intelligent Systems and Computing, 722, pp.589~594 (2018)[DOI]
  63. Park, S., "1H3-1 Affective qualities in in-vehicle interiors? a case study on instrument panel via semantic network analysis", 人間工学, 55(Supplement), pp.1H3-1~ (2019)
  64. Partala, T., "Understanding the most satisfying and unsatisfying user experiences : Emotions, psychological needs, and context", Interacting with Computers, 24(1), pp.25~34 (2012)
  65. Pradhan, A., "Accessibility came by accident: use of voice-controlled intelligent personal assistants by people with disabilities", Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems, 459 (2018)
  66. Pyae, A., "Investigating the usability and user experiences of voice user interface: a case of Google home smart speaker", Proceedings of the 20th International Conference on Human-Computer Interaction with Mobile Devices and Services Adjunct, pp.127~131 (2018)
  67. Pyae, A., "Investigating differences between native English and non-native English speakers in interacting with a voice user interface:A case of Google Home", Proceedings of the 30th Australian Conference on Computer-Human Interaction, pp.548553~ (2018)
  68. Rhie, Y. L., "A Semantic Network Analysis as a Method for Understanding Qualitative User Experience in Product Interactions", pp.2017
  69. Ye Lim Rhie, "Evaluating Representativeness of Qualitative Text Data in Identifying UX Issues", International Journal of Human–Computer Interaction, 33(11), pp.868~881 (2017), Informa UK Limited[DOI]
  70. "One year after Amazon introduced Echo, half a million people have told Alexa, ‘I love you.’" (2015)[URL]
  71. Rodolitz, J., "Accessibility of VoiceActivated Agents for People who are Deaf or Hard of Hearing", The Journal on Technology and Persons with Disabilities (2019)
  72. Umair Saad, "A model to measure QoE for virtual personal assistant", Multimedia Tools and Applications, 76(10), pp.12517~12537 (2017), Springer Science and Business Media LLC[DOI]
  73. Sali, S., "Getting a word in: adding artificial pauses to natural language interaction", Proceedings of the International Conference on the Foundations of Digital Games, pp.57~64 (2012)
  74. Sand, F. van de, "User Experience Is Brand Experience", pp.71~93 (2020), Springer
  75. Satyanarayanan, M., "Pervasive computing : Vision and challenges", IEEE Personal Communications, 8(4), pp.10~17 (2001)
  76. Scott, J., "The SAGE handbook of social network analysis" (2011), SAGE publications
  77. Simon, S. J., "User acceptance of voice recognition technology : an empirical extension of the technology acceptance model", Journal of Organizational and End User Computing(JOEUC), 19(1), pp.24~50 (2007)
  78. Valente, T. W., "How correlated are network centrality measures?", Connections (Toronto, Ont.), 28(1), pp.16~ (2008)
  79. Venkatesh, "Consumer Acceptance and Use of Information Technology: Extending the Unified Theory of Acceptance and Use of Technology", MIS Quarterly, 36(1), pp.157~ (2012), JSTOR[DOI]
  80. Venkatesh, "User Acceptance of Information Technology: Toward a Unified View", MIS Quarterly, 27(3), pp.425~ (2003), JSTOR[DOI]
  81. Von Helversen, B., "Influence of consumer reviews on online purchasing decisions in older and younger adults", Decision Support Systems, 113, pp.1~10 (2018)
  82. Vtyurina, A., "Exploring the role of conversational cues in guided task support with virtual assistants", Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems, 208 (2018)
  83. Webster, J., "The dimensionality and correlates of flow in human-computer interactions", Computers in Human Behavior, 9(4), pp.411~426 (1993)

관련 논문AI추천

이 간행물의 다른 논문 (4건)