RBM을 이용한 언어의 분산 표상화
RBM-based distributed representation of language
0
조회
0
다운로드
0
인용
0
참고문헌
키워드
초록
연결주의 모델은 계산주의적 관점에서 언어 처리를 연구하는 한 가지 접근법이다. 그리고 연결주의 모델 연구를 진행하는데 있어서 표상(representation)을 구축하는 것은, 모델의 학습 수준 및 수행 능력을 결정한다는 점에서 모델의 구조를 만드는 것만큼이나 중요한 일이다. 연결주의 모델은 크게 지역 표상(localist representation)과 분산 표상(distributed representation)이라는 두 가지 서로 다른 방식으로 표상을 구축해 왔다. 하지만 종래 연구들에서 사용된 지역 표상은 드문 목표 활성화 값을 갖고 있는 출력층의 유닛이 불활성화하는 제한점을, 그리고 과거의 분산 표상은 표상된 정보의 불투명성에 의한 결과 확인의 어려움이라는 제한점을 갖고 있었으며 이는 연결주의 모델 연구 전반의 제한점이 되어 왔다. 본 연구는 이와 같은 과거의 표상 구축의 제한점에 대하여, 제한된 볼츠만 머신(restricted Boltzmann machine)이 갖고 있는 특징인 정보의 추상화를 활용하여 지역 표상을 가지고 분산 표상을 유도하는 새로운 방안을 제시하였다. 결과적으로 본 연구가 제안한 방법은 정보의 압축과 분산 표상을 지역 표상으로 역변환하는 방안을 활용하여 종래의 표상 구축 방법이 갖고 있는 문제를 효과적으로 해결함을 보였다.
Abstract
The connectionist model is one approach to studying language processing from a computational perspective. And building a representation in the connectionist model study is just as important as making the structure of the model in that it determines the level of learning and performance of the model. The connectionist model has been constructed in two different ways: localist representation and distributed representation. However, the localist representation used in the previous studies had limitations in that the unit of the output layer having a rare target activation value is inactivated, and the past distributed representation has the limitation of difficulty in confirming the result by the opacity of the displayed information. This has been a limitation of the overall connection model study. In this paper, we present a new method to induce distributed representation with local representation using abstraction of information, which is a feature of restricted Boltzmann machine, with respect to the limitation of such representation of the past. As a result, our proposed method effectively solves the problem of conventional representation by using the method of information compression and inverse transformation of distributed representation into local representation.
참고문헌 (17)
- 유희조, "속성적 관점에 기반한단어 재인시의 의미 처리 가능성 연구: 연결주의 모델링", 한국심리학회지: 인지 및 생물, 27(4), pp.613~638 (2015), 한국인지및생물심리학회
- George S. Cree, "Distinctive features hold a privileged status in the computation of word meaning: Implications for theories of semantic memory.", Journal of Experimental Psychology: Learning, Memory, and Cognition, 32(4), pp.643~658 (2006), American Psychological Association (APA)[DOI]
- Katia Dilkina, "A single-system account of semantic and lexical deficits in five semantic dementia patients", Cognitive Neuropsychology, 25(2), pp.136~164 (2008), Informa UK Limited[DOI]
- Dilkina, K., "Are There Mental Lexicons? The Role of Semantics in Lexical Decision", Brain Research, 1365, pp.66~81 (2010)[DOI]
- Michael W. Harm, "Phonology, reading acquisition, and dyslexia: Insights from connectionist models.", Psychological Review, 106(3), pp.491~528 (1999), American Psychological Association (APA)[DOI]
- Hinton, G., "A Practical Guide to Training Restricted Boltzmann Machines A Practical Guide to Training Restricted Boltzmann Machines", Computer, 9(1) (2010)[DOI]
- Geoffrey E. Hinton, "Lesioning an attractor network: Investigations of acquired dyslexia.", Psychological Review, 98(1), pp.74~95 (1991), American Psychological Association (APA)[DOI]
- Kawamoto, A., "Nonlinear dynamics in the resolution of lexical ambiguity: A parallel distributed processing account", Journal of Memory and Language, 32, pp.474~516 (1993)
- Laszlo, S., "A neurally plausible parallel distributed processing model of event-related potential word reading data", Brain and Language, 120, pp.271~281 (2012)[DOI]
- McClelland, J., "An interactive activation model of context effects in letter perception : I. An account of basic findings", Psychological Review (1981)
- Mikolov, T., "Efficient Estimation of Word Representations in Vector Space", In Proceedings of the International Conference on Learning Representations, pp.1~12 (2013)
- David C. Plaut, "Structure and Function in the Lexical System: Insights from Distributed Models of Word Reading and Lexical Decision", Language and Cognitive Processes, 12(5-6), pp.765~806 (1997), Informa UK Limited[DOI]
- Plaut, D. C., "Understanding normal and impaired word reading: computational principles in quasi-regular domains", Psychological Review, 103, pp.56~115 (1996)
- Plaut, D. C., "Deep dyslexia : A case study of connectionist neuropsychology", Cognitive Neuropsychology, 10, pp.377~500 (1993)
- Timothy T. Rogers, "Structure and Deterioration of Semantic Memory: A Neuropsychological and Computational Investigation.", Psychological Review, 111(1), pp.205~235 (2004), American Psychological Association (APA)[DOI]
- Seidenberg, M. S., "A distributed, developmental model of word recognition and naming", Psychological Review, 96, pp.523~568 (1989)
- Smolensky, P., "Parallel Distributed Processing Explorations in the Microstructure of Cognition Vol. 1", pp.194~281 (1986)