본문 바로가기
KERD
논문KCI 등재

MACVTL: Improving Efficiency of Mood Analysis from Text & Facial Cues Using Cross Validation and Transfer Learning

0

조회

0

다운로드

0

인용

0

참고문헌

키워드

초록

Accurate analysis of a person’s mood can assist medical psychiatry personnel to assist and improve personal well-being. In order to determine a person’s mood a wide variety of inputs can be used, these inputs include but are not limited to, the person’s social media patterns, facial patterns, ecommerce buying patterns, etc. Analysis of these patterns individually can assist in development of a moderate accuracy psycho-analysis system. Thus, to improve the accuracy and efficiency of mood analysis, this text proposes a deep learning model based on transfer learning, that combines training information from social media activity, and visual actions in order to analyze the person’s state-of-mind. The proposed model is compared with individual algorithms that detect person’s mood, and an accuracy improvement of 12% was achieved. The model is also compared with similar hybrid algorithms that combine multiple psychiatric ques, and it is observed that the proposed model is 8% more effective than these systems due to incorporation of cross validation and transfer learning.

Abstract

Accurate analysis of a person’s mood can assist medical psychiatry personnel to assist and improve personal well-being. In order to determine a person’s mood a wide variety of inputs can be used, these inputs include but are not limited to, the person’s social media patterns, facial patterns, ecommerce buying patterns, etc. Analysis of these patterns individually can assist in development of a moderate accuracy psycho-analysis system. Thus, to improve the accuracy and efficiency of mood analysis, this text proposes a deep learning model based on transfer learning, that combines training information from social media activity, and visual actions in order to analyze the person’s state-of-mind. The proposed model is compared with individual algorithms that detect person’s mood, and an accuracy improvement of 12% was achieved. The model is also compared with similar hybrid algorithms that combine multiple psychiatric ques, and it is observed that the proposed model is 8% more effective than these systems due to incorporation of cross validation and transfer learning.

참고문헌 (47)

  1. Akhapkin, R. V., "Recognition of facial emotion expressions in patients with depressive disorders : a prospective, observational study", Neurology and Therapy, 10, pp.225~234 (2021)
  2. Aloshban, N., "What you say or how you say it? depression detection through joint modeling of linguistic and acoustic aspects of speech", Cognitive Computation, 14(5), pp.1585~1598 (2022)
  3. Behera, A., "Associating Facial Expressions and Upper-Body Gestures with Learning Tasks for Enhancing Intelligent Tutoring Systems", International Journal of Artificial Intelligence in Education, 30, pp.236~270 (2020)
  4. Bekendam, M. T., "Facial expressions of emotions during pharmacological and exercise stress testing : The role of myocardial ischemia and cardiac symptoms", International Journal of Behavioral Medicine, 28, pp.692~704 (2021)
  5. Bhaskar, J., "Hybrid approach for emotion classification of audio conversation based on text and speech mining", Procedia Computer Science, 46, pp.635~643 (2015)
  6. Chancellor, S., "Methods in predictive techniques for mental health status on social media : a critical review", NPJ digital medicine, 3(1), pp.43~ (2020)
  7. Chen, F., "Agomelatine softens depressive-like behavior through the regulation of autophagy and apoptosis", BioMed Research International, 2021 (2021)
  8. Cohn, J. F., "Detecting depression from facial actions and vocal prosody", 2009 3rd International Conference on Affective Computing and Intelligent Interaction and Workshops, pp.1~7 (2009), IEEE
  9. Dadiz, B. G., "Detecting depression in videos using uniformed local binary pattern on facial features", Computational Science and Technology: 5th ICCST 2018, pp.413~422 (2018), Springer
  10. Ebrahimi Kahou, S., "Recurrent neural networks for emotion recognition in video", Proceedings of the 2015 ACM on international conference on multimodal interaction, pp.467~474 (2015)
  11. Ellement, B., "Social cognition in cervical dystonia : phenotype and relationship to anxiety and depression", European Journal of Neurology, 28(1), pp.98~107 (2021)
  12. Francese, R., "A user-centered approach for detecting emotions with low-cost sensors", Multimedia Tools and Applications, 79, pp.35885~35907 (2020)
  13. Gaebel, W., "Facial expression and emotional face recognition in schizophrenia and depression", European archives of psychiatry and clinical neuroscience, 242, pp.46~52 (1992)
  14. Ghaemi, S. N., "Digital depression : a new disease of the millennium?", Acta Psychiatrica Scandinavica, 141(4), pp.356~361 (2020)
  15. He, L., "Automatic depression recognition using CNN with attention mechanism from videos", Neurocomputing, 422, pp.165~175 (2021)
  16. Islam, M. R., "Depression detection from social network data using machine learning techniques", Health information science and systems, 6, pp.1~12 (2018)
  17. Kaiser, T., "Unraveling the comorbidity of depression and anxiety in a large inpatient sample : Network analysis to examine bridge symptoms", Depression and Anxiety, 38(3), pp.307~317 (2021)
  18. Kim, J., "Machine learning for mental health in social media : bibliometric study", Journal of Medical Internet Research, 23(3), pp.e24870~ (2021)
  19. Kurpukdee, N., "Speech emotion recognition using convolutional long short-term memory neural network and support vector machines", 2017 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC), pp.1744~1749 (2017), IEEE
  20. Li, Z., "Major depressive disorder : advances in neuroscience research and translational applications", Neuroscience bulletin, 37, pp.863~880 (2021)
  21. Lin, C. Y., "Mediated roles of generalized trust and perceived social support in the effects of problematic social media use on mental health : A cross-sectional study", Health Expectations, 24(1), pp.165~173 (2021)
  22. Mahato, S., "Detection of depression and scaling of severity using six channel EEG data", Journal of medical systems, 44, pp.1~12 (2020)
  23. Malaeb, D., "Problematic social media use and mental health(depression, anxiety, and insomnia)among Lebanese adults : Any mediating effect of stress?", Perspectives in psychiatric care, 57(2), pp.539~549 (2021)
  24. Mamtora, R., "Improving Accuracy of Sentiment Analysis for Depression Recommendation using Multi-Domain Fuzzy Rules", International Journal of Innovative Technology and Exploring Engineering (IJITEE), 9(4) (2020)
  25. Meng, H, "Depression recognition based on dynamic facial and vocal expression features using partial least square regression", Proceedings of the 3rd ACM international workshop on Audio, 2013, 21~30
  26. Nasir, M, "Multimodal and multiresolution depression detection from speech and facial landmark features", Proceedings of the 6th international workshop on audio, 2016, 43~50
  27. Naslund, J. A., "Social media and mental health : benefits, risks, and opportunities for research and practice", Journal of technology in behavioral science, 5, pp.245~257 (2020)
  28. Nicolini, P., "Different dimensions of social support differentially predict psychological well-being in late life : Opposite effects of perceived emotional support and marital status on symptoms of anxiety and of depression in older outpatients in Italy", Psychogeriatrics, 21(1), pp.42~53 (2021)
  29. Pampouchidou, A., "Automated facial video-based recognition of depression and anxiety symptom severity : cross-corpus validation", Machine Vision and Applications, 31(4), pp.30~ (2020)
  30. Priya, A., "Predicting anxiety, depression and stress in modern life using machine learning algorithms", Procedia Computer Science, 167, pp.1258~1267 (2020)
  31. Rahman, M., "An efficient hybrid system for anomaly detection in social networks", Cybersecurity, 4(1), pp.1~11 (2021)
  32. Rescigno, M., "Personalized models for facial emotion recognition through transfer learning", Multimedia Tools and Applications, 79, pp.35811~35828 (2020)
  33. Risch, J., "Deep Learning-Based Approaches for Sentiment Analysis" (2020), Springer
  34. Singh, M., "Sentiment analysis on the impact of coronavirus in social life using the BERT model", Social Network Analysis and Mining, 11(1), pp.33~ (2021)
  35. Tao, X., "Detecting depression using an ensemble classifier based on Quality of Life scales", Brain Informatics, 8, pp.1~15 (2021)
  36. Trotzek, M., "Utilizing neural networks and linguistic metadata for early detection of depression indications in text sequences", IEEE Transactions on Knowledge and Data Engineering, 32(3), pp.588~601 (2018)
  37. Tuncer, T., "A novel facial image recognition method based on perceptual hash using quintet triple binary pattern", Multimedia Tools and Applications, 79, pp.29573~29593 (2020)
  38. Ullah, A., "Fusion of machine learning and privacy preserving for secure facial expression recognition", Security and Communication Networks, 2021, pp.1~12 (2021)
  39. Vitriol, V., "Latent Class Analysis in Depression, Including Clinical and Functional Variables: Evidence of a Complex Depressive Subtype in Primary Care in Chile" (2021), Depression research and treatment
  40. Volk, G. F., "Role of Body Dysmorphic Disorder in Patients with Postparalytic Facial Synkinesis", The Laryngoscope, 131(9), pp.E2518~E2524 (2021)[DOI]
  41. Wang, P., "A novel consumer purchase behavior recognition method using ensemble learning algorithm", Mathematical Problems in Engineering, 2020, pp.1~10 (2020)
  42. Wang, X., "A depression detection model based on sentiment analysis in micro-blog social network", Trends and Applications in Knowledge Discovery and Data Mining: PAKDD 2013 International Workshops: DMApps, DANTH, QIMIE, BDM, CDA, CloudSD, Gold Coast, QLD, pp.201~213 (2013), Springer Berlin Heidelberg
  43. Yoon, S., "Reading simple and complex facial expressions in patients with major depressive disorder and anxiety disorders", Psychiatry and Clinical Neurosciences, 70(3), pp.151~158 (2016)
  44. Youssef, L., "Social media use disorder and alexithymia : any association between the two? Results of a cross-sectional study among Lebanese adults", Perspectives in Psychiatric Care, 57(1), pp.20~26 (2021)
  45. Zhou, X., "Learning content-adaptive feature pooling for facial depression recognition in videos", Electronics Letters, 55(11), pp.648~650 (2019)
  46. Zhou, X., "Visually interpretable representation learning for depression recognition from facial images", IEEE Transactions on Affective Computing, 11(3), pp.542~552 (2018)
  47. Zucco, C., "Sentiment analysis for mining texts and social networks data : Methods and tools", Wiley Interdisciplinary Reviews : Data Mining and Knowledge Discovery, 10(1), pp.e1333~ (2020)

관련 논문AI추천

이 간행물의 다른 논문 (4건)