AI as a Pair Programming Buddy: Exploring Design Considerations for Human-AI Collaboration
0
조회
0
다운로드
2
인용
0
참고문헌
키워드
초록
This qualitative study delves into the collaboration experiences of programmers with Artificial Intelligence (AI) in programming, with a focus on key aspects like usability, workload, trust, and learning. Programmers were assigned coding tasks using Copilot, an AI tool, and interviewed to gather their feedback. Effective communication emerges as a pivotal factor in ensuring successful human-AI collaboration. The AI system should grasp intentions, offer accurate code suggestions, provide transparent explanations, and handle errors akin to a human partner. To enhance the programmer's experience with AI-assisted pair programming, the study presents a framework that prioritizes improved communication, code suggestions, explanations, and error resolution. This framework promises to yield a more productive and rewarding experience with AI. The research provides valuable user perspectives, deepening our understanding of AI-assisted pair programming dynamics. Moreover, it lays the groundwork for future AI developments, particularly in supporting programmers. Given the continuous advancements in AI, human-AI collaboration holds tremendous potential to boost productivity and innovation in software development. Leveraging AI's capabilities in programming can unlock greater efficiency and creativity, heralding a promising future for AI integration in the field. Through this investigation, we gain insights that pave the way for a more seamless and productive collaboration between programmers and AI systems. This contributes to the advancement of AI applications in programming domains.
Abstract
This qualitative study delves into the collaboration experiences of programmers with Artificial Intelligence (AI) in programming, with a focus on key aspects like usability, workload, trust, and learning. Programmers were assigned coding tasks using Copilot, an AI tool, and interviewed to gather their feedback. Effective communication emerges as a pivotal factor in ensuring successful human-AI collaboration. The AI system should grasp intentions, offer accurate code suggestions, provide transparent explanations, and handle errors akin to a human partner. To enhance the programmer's experience with AI-assisted pair programming, the study presents a framework that prioritizes improved communication, code suggestions, explanations, and error resolution. This framework promises to yield a more productive and rewarding experience with AI. The research provides valuable user perspectives, deepening our understanding of AI-assisted pair programming dynamics. Moreover, it lays the groundwork for future AI developments, particularly in supporting programmers. Given the continuous advancements in AI, human-AI collaboration holds tremendous potential to boost productivity and innovation in software development. Leveraging AI's capabilities in programming can unlock greater efficiency and creativity, heralding a promising future for AI integration in the field. Through this investigation, we gain insights that pave the way for a more seamless and productive collaboration between programmers and AI systems. This contributes to the advancement of AI applications in programming domains.
참고문헌 (73)
- L. Williams, "Strengthening the case for pair programming", IEEE software, 17(4), pp.19~25 (2000)
- 양권우, "짝 프로그래밍을 이용한 소프트웨어 교육이 학습동기 및 학업성취도에 미치는 영향", 한국지식정보기술학회 논문지, 15(1), pp.57~65 (2020), 한국지식정보기술학회[DOI]
- A. Begel, "Pair programming: what’s in it for me?", Proceedings of the Second ACM-IEEE international symposium on Empirical software engineering and measurement, pp.120~128 (2008)
- K. M. Lui, "Pair programming productivity : Novice–novice vs. expert–expert", International Journal of Human-Computer Studies, 64(9), pp.915~925 (2006)
- Y. Li, "Competition-level code generation with AlphaCode", Science, 378(6624), pp.1092~1097 (2022)
- P. Vaithilingam, "Expectation vs. experience:Evaluating the usability of code generation tools powered by large language models", CHI Conference on Human Factors in Computing Systems Extended Abstracts, pp.1~7 (2022)
- J. Finnie-Ansley, "The robots are coming: Exploring the implications of openai codex on introductory programming", Proceedings of the 24th Australasian Computing Education Conference, pp.10~19 (2022)
- "Your AI pair programmer" (2022)[URL]
- X. Zhai, "ChatGPT user experience : Implications for education" (2022), SSRN
- A. Ziegler, "Productivity assessment of neural code completion", Proceedings of the 6th ACM SIGPLAN International Symposium on Machine Programming, pp.21~29 (2022)
- M. Jaworski, "Study of software developers’ experience using the Github Copilot Tool in the software development process", arXiv : 2301. 04991, ~ (2023)
- B. Shackel, "Usability–context, framework, definition, design and evaluation", Interacting with computers, 21(5-6), pp.339~346 (2009)
- D. Stotts, "Support for distributed pair programming in the transparent video facetop", Extreme Programming and Agile Methods-XP, 2004, ~
- N. Bevan., "Classifying and selecting UX and usability measures", International Workshop on Meaningful Measures: Valid Useful User Experience Measurement, 11 (2008), Institute of Research in Informatics of Toulouse (IRIT)
- L. Williams, "Eleven guidelines for implementing pair programming in the classroom", Agile 2008 Conference, pp.445~452 (2008), IEEE
- D. Szafron, "An experiment to measure the usability of parallel programming systems", Concurrency: Practice and Experience, 8(2), pp.147~166 (1996)
- G. V. Wilson, "Assessing the usability of parallel programming systems: The Cowichan problems", Programming Environments for Massively Parallel Distributed Systems:Working Conference of the IFIP WG, pp.183~ (1994)
- Usability issues in the design of novice programming systems, "J. F. Pane", Carnegie-Mellon University. Department of Computer Science, 96~132 (1996)
- L. Lin, "Applying human factors to the design of medical equipment : patient-controlled analgesia", Journal of clinical monitoring and computing, 14, pp.253~263 (1998)
- I. M. Félix, "A study to build a new Visual Programming system:Fixed or contextual menu?", 2019 IEEE Frontiers in Education Conference (FIE), pp.1~8 (2019)
- K. Dönmez, "Handling the pseudo pilot assignment problem in air traffic control training by using NASA TLX", Journal of Air Transport Management, 89 (2020)
- J. Dominic, "The human experience of comprehending source code in virtual reality", Empirical Software Engineering, 27(7), pp.173~ (2022)
- B. J. Grosz, "Attention, intentions, and the structure of discourse", Computational linguistics, 12(3), pp.175~204 (1986)
- P. R. Cohen, "Teamwork", Nous, 25(4), pp.487~512 (1991)
- M. Tomasello, "Understanding and sharing intentions : The origins of cultural cognition", Behavioral and Brain Sciences, 28(5), pp.675~691 (2005)
- R. Singh, "Combining gaze and AI planning for online human intention recognition", Artificial Intelligence, 284 (2020)
- W. Wang, "Human intention prediction in human-robot collaborative tasks", Companion of the 2018ACM, 2018, 279~280
- D. B. Wolf, "Interaction styles in work relationships", Doctoral, pp.2019
- S. S. Sundar, "Rise of machine agency : A framework for studying the psychology of human–AI interaction(HAII)", Journal of Computer-Mediated Communication, 25(1), pp.74~88 (2020)
- B. Hanks, "Student attitudes toward pair programming", Proceedings of the 11th annual SIGCSE conference on Innovation and technology in computer science education (2006)
- M. Zarb, "Breaking the communication barrier : guidelines to aid communication within pair programming", Computer science education, 25(2), pp.120~151 (2015)
- J. Chong, "The social dynamics of pair programming", 29th International Conference on Software Engineering(ICSE'07) (2007)
- A. J. Dick, "Paired programming and personality traits", XP2002, ~ (2002)
- S. Barke, "Grounded copilot: How programmers interact with code-generating models", Proceedings of the ACM on Programming Languages, pp.85~111 (2023)
- D. Jayagopal, "Exploring the learnability of program synthesizers by novice programmers", Proceedings of the 35th Annual ACM Symposium on User Interface Software and Technology, pp.1~15 (2022)
- C. Bird, "Taking flight with Copilot : Early insights and opportunities of AI-powered pair-programming tools", Queue, 20(6), pp.35~57 (2022)
- M. Kazemitabaar, "Studying the effect of AI code generators on supporting novice learners in introductory programming", Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems, pp.1~23 (2023)
- N. Perry, "Do users write more insecure code with AI assistants?", arXiv preprint arXiv : 2211. 03622, ~ (2022)
- A. Sarkar, "What is it like to program with artificial intelligence?", arXiv preprint arXiv : 2208. 06213, ~ (2022)
- M. Crouch, "The logic of small samples in interview-based qualitative research", Social science information, 45(4), pp.483~499 (2006)
- K. P. Subedi, "Determining the sample in qualitative research", Online Submission, 4, pp.1~13 (2021)
- M. Mason, "Sample size and saturation in PhD studies using qualitative interviews", Forum qualitative Sozialforschung, 2010, pp.11(3), ~
- A. Bhattacherjee, "Understanding information systems continuance : An expectation-confirmation model", MIS Quarterly, 25(3), pp.351~370 (2001)
- F. D. Davis, "User acceptance of computer technology : A comparison of two theoretical models", Management Science, 35(8), pp.982~1003 (1989)
- K. Nonose, "An exploratory study: a measure of workload associated with teamwork", Cognition, Technology & Work, 18(2), pp.351~360 (2015)
- E. Salas, "Is there a ‘Big Five’ in teamwork?", Small Group Research, 36(5), pp.555~599 (2005)
- 이희준, "인공지능 스피커(AI 스피커)에 대한 사용자 인식과 이용 동기 요인 연구", 한국콘텐츠학회 논문지, 19(3), pp.138~154 (2019), 한국콘텐츠학회[DOI]
- S. T. Foster, "Managing quality : Integrating the supply chain" (2022), John Wiley & Sons
- A. Awasthi, "A hybrid approach integrating Affinity Diagram, AHP and fuzzy TOPSIS for sustainable city logistics planning", Applied Mathematical Modelling, 36(2), pp.573~584 (2012)
- S. Takai, "A use of subjective clustering to support affinity diagram results in customer needs analysis", Concurrent Engineering, 18(2), pp.101~109 (2010)
- S. Imai, "Is github copilot a substitute for human pair-programming? an empirical study", Proceedings of the ACM, 2022, 319~321
- C. Brown, "Sorry to bother you: Designing bots for effective recommendations", 2019 IEEE, 2019, 54~58
- R. Buettner, "Cognitive workload of humans using artificial intelligence systems : towards objective measurement applying eye-tracking technology", Advances in Artificial Intelligence, 8077, pp.37~48 (2013)
- 정혜욱, "A Study on Coding Education for Non-Computer Majors Using Programming Error List", The International Journal of Advanced Culture Technology, 9(1), pp.203~209 (2021), 국제문화기술진흥원
- A. Dafoe, "Open problems in cooperative AI", arXiv preprint arXiv : 2012. 08630, ~ (2020)
- D. Tsompanoudi, "Evaluating the effects of scripted distributed pair programming on student performance and participation", IEEE Transactions on Education, 59(1), pp.24~31 (2016)
- J-Y. Mao, "The state of user-centered design practice", Communications of the ACM, 48(3), pp.105~109 (2005)
- O. E. Llantos, "Learning interactions as it evolves in a social learning management system", 2021Eighth International Conference on Social Network Analysis, Management and Security (SNAMS) (2021)
- T. Jokela, "Usability assessment of an extreme programming project: Close co-operation with the customer does not equal to good usability", International Conference on Product Focused Software Process Improvement. Berlin (2004), Springer Berlin Heidelberg
- D. Tsompanoudi, "Exploring the effects of collaboration scripts embedded in a distributed pair programming system", Proceedings of the 18th ACM conference on Innovation and technology in computer science education, pp.225~230 (2013)
- L. L. Constantine, "Process agility and software usability : Toward lightweight usage-centered design", Information Age, 8(8), pp.1~10 (2002)
- A. F. C. Morales, "Development of the NASA-TLX multi equation tool to assess workload", International Journal of Combinatorial Optimization Problems and Informatics, 11(1), pp.50~ (2020)
- R. A. Grier, "How high is high? A meta-analysis of NASA-TLX global workload scores", Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 59(1), pp.1727~1731 (2015)
- A. Steinmaurer, "Interactive Mobile Communication, Technologies and Learning", pp.547~558 (2021), Springer International Publishing
- R. Couceiro, "Pupillography as indicator of programmers'mental effort and cognitive overload", 201949th Annual IEEE, 2019, 638~644[DOI]
- K. Setiya, "Intention, The Stanford Encyclopedia of Philosophy" (2009)
- Unit-3 analysis of human action, "A. Pinheiro", IGNOU, ~ (1970)
- J. S. Bruner, "Intention in the structure of action and interaction", Advances in infancy research, 1, pp.41~56 (1981)
- A. N. Meltzoff, "Understanding the intentions of others: re-enactment of intended acts by 18-month-old children", Developmental psychology, 31(5), pp.838~ (1995)
- D. Premack, "Does the chimpanzee have a theory of mind?", Behavioral and brain sciences, 1(4), pp.515~526 (1978)
- I. A. Apperly, "Do humans have two systems to track beliefs and belief-like states?", Psychological review, 116(4), pp.953~ (2009)
- M. El Ayadi, "Survey on speech emotion recognition:Features, classification schemes, and databases", Pattern recognition, 44(3), pp.572~587 (2011)