프롬프트의 언어학적 특성과 상관관계 분석: LFTK (Linguistic Feature ToolKit) 기반 연구
Linguistic Features of Prompts and Correlation Analysis: A Study Based on LFTK (Linguistic Feature ToolKit)
0
조회
0
다운로드
0
인용
0
참고문헌
키워드
초록
This study investigates the nature of good and bad prompts extracted from Hugging Face datasets using the LFTK (Linguistic Feature Tool Kit) package and analyzes the correlation of the linguistic features of good and bad prompts through Spearman’s Rank Correlation Analysis. Then, this study is to provide efficient prompting strategies from the perspective of linguistics, in order to reduce hallucination generated by Artificial Intelligence which contains false or misleading information. The results of this paper reveal that certain linguistic features, such as linguistic units (i.e., characters, syllables, words, categories), text difficulty, and lexical diversity, can affect the quality of prompts. Specifically, the length of sentences and words, word difficulty, the use of nouns and determiners, and the frequency of commonly used words are found to be significant factors when prompting. In conclusion, this paper may contribute to providing Generative AI users with guidelines on how to make linguistically well-crafted prompts with high quality.
Abstract
This study investigates the nature of good and bad prompts extracted from Hugging Face datasets using the LFTK (Linguistic Feature Tool Kit) package and analyzes the correlation of the linguistic features of good and bad prompts through Spearman’s Rank Correlation Analysis. Then, this study is to provide efficient prompting strategies from the perspective of linguistics, in order to reduce hallucination generated by Artificial Intelligence which contains false or misleading information. The results of this paper reveal that certain linguistic features, such as linguistic units (i.e., characters, syllables, words, categories), text difficulty, and lexical diversity, can affect the quality of prompts. Specifically, the length of sentences and words, word difficulty, the use of nouns and determiners, and the frequency of commonly used words are found to be significant factors when prompting. In conclusion, this paper may contribute to providing Generative AI users with guidelines on how to make linguistically well-crafted prompts with high quality.
참고문헌 (18)
- 김소연, "생성형 AI의 신뢰도에 대한 탐색적 연구", 인터넷정보학회논문지, 25(1), pp.49~60 (2024)
- 손민주, "입력 텍스트의 언어학적 특성이 자연어 생성 모델의 환각 현상에 미치는 영향 탐구", 석사, pp.2024
- 양용준, "영어와 한국어의 한정사(Determiners)에 관한 비교분석", 영어영문학, 18(2), pp.145~165 (2013)
- 황이수, "수학능력시험 영어 지문의 텍스트 요인과 문항 난이도의 상관관계 분석 : 통사적 복잡성을 중심으로", 영어영문학연구, 46(1), pp.265~283 (2020)
- Brysbaert, M., "Test-based age-of-acquisition norms for 44 thousand English word meanings", Behavior research methods, 49, pp.1520~1523 (2017)
- Brysbaert, M., "Moving beyond Kučera and Francis : A critical evaluation of current word frequency norms and the introduction of a new and improved word frequency measure for American English", Behavior Research Methods, 41(4), pp.977~990 (2009)
- "Gemini for Google Workspace - Prompting Guide 101" (2024)[URL]
- Kuperman, V., "Age-of-acquisition ratings for 30, 000 English words", Behavior Research Methods, 44, pp.978~990 (2012)
- Lee, B. W., "LFTK: Handcrafted features in computational linguistics", Paper presented at Proceedings of the 18th Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2023) (2023)
- "Traditional readability formulas compared for English" (2023)[URL]
- Leidinger, A., "The language of prompting: What linguistic properties make a prompt successful?", Paper presented at Findings of the Association for Computational Linguistics: EMNLP 2023 (2023)
- Lin, Z., "How to write effective prompts for large language models", Nature Human Behaviour, 8(4), pp.611~615 (2024)
- Marvin, G., "Prompt engineering in large language models", Paper presented at the International conference on data intelligence and cognitive informatics (2023)
- "Exploring the relationship between LLM hallucinations and prompt linguistic nuances: Readability, formality, and concreteness" (2023)[URL]
- Rovai, A. P., "Social science research design and statistics: A practitioner's guide to research methods and IBM SPSS" (2013), Watertree Press LLC
- Van Heuven, W. J., "SUBTLEX-UK : A new and improved word frequency database for British English", Quarterly Journal of Experimental Psychology, 67(6), pp.1176~1190 (2014)
- "A prompt pattern catalog to enhance prompt engineering with chatgpt" (2023)[URL]
- Zamfirescu-Pereira, J. D., "Why Johnny can’t prompt: How non-AI experts try (and fail) to design LLM prompts", Paper presented at Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems (2023)