개미 집단 시스템에서 TD-오류를 이용한 강화학습 기법
A Reinforcement Learning Method using TD-Error in Ant Colony System
0
조회
0
다운로드
0
인용
0
참고문헌
키워드
초록
강화학습에서 temporal-credit 할당 문제 즉, 에이전트가 현재 상태에서 어떤 행동을 선택하여 상태전이를 하였을 때 에이전트가 선택한 행동에 대해 어떻게 보상(reward)할 것인가는 강화학습에서 중요한 과제라 할 수 있다. 본 논문에서는 조합최적화(hard combinational optimization) 문제를 해결하기 위한 새로운 메타 휴리스틱(meta heuristic) 방법으로, greedy search뿐만 아니라 긍정적 반응의 탐색을 사용한 모집단에 근거한 접근법으로 Traveling Salesman Problem(TSP)를 풀기 위해 제안된 Ant Colony System(ACS) Algorithms에 Q-학습을 적용한 기존의 Ant-Q 학습방법을 살펴보고 이 학습 기법에 다양화 전략을 통한 상태전이와 TD-오류를 적용한 학습방법인 Ant-TD 강화학습 방법을 제안한다. 제안한 강화학습은 기존의 ACS, Ant-Q학습보다 최적해에 더 빠르게 수렴할 수 있음을 실험을 통해 알 수 있었다.
Abstract
Reinforcement learning takes reward about selecting action when agent chooses some action and did state transition in present state. this can be the important subject in reinforcement learning as temporal-credit assignment problems. In this paper, by new meta heuristic method to solve hard combinational optimization problem, examine Ant-Q learning method that is proposed to solve Traveling Salesman Problem (TSP) to approach that is based for population that use positive feedback as well as greedy search. And, suggest Ant-TD reinfoecement learning method that apply state transition through diversification strategy to this method and TD-error. We can show through experiments that the reinforcement learning method proposed in this paper can find out an optimal solution faster than other reinforcement learning method like ACS and Ant-Q learning.
참고문헌 (15)
- "An investigation of some properties of an ant algorithm", pp.509~520 (1992), Proceediings of the Parallel Parallel Problem Solving from Nature Conference (PPSn'92) R. Manner and B. Manderick (Eds.) Elsevier Publishing
- "Distributed optimization by ant colonies Proceedings of ECAL'91 - European Conference of Artificial Life", pp.134~144 (1991), Elsevier Publishing
- "Learning from Delayed Rewards", king's college, pp.1989~
- "Efficient reinforcement learning", pp.88~97 (1994)
- "IEEE Transactions on Systems", pp.357~365 (1993)
- "Solving symmetric and asymmetric TSPs by ant colonies", pp.622~627 (1996)
- "Ant Colony System：A Cooperative Learning approach to the Traveling Salesman Problem", 1(1) (1997)
- "Ant-Q：a reinforcement learning approach to the traveling salesman problem", pp.252~260 (1995)
- "A study of some properties of Ant-Q", pp.656~665 (1996)
- "The ant system： optimization by a colony of cooperation agents", 26(2), pp.29~41 (1996)
- "Ant Algorithms for Discrete Optimization", 5(3), pp.137~172 (1999)
- "Ant Colonies for the Traveling Salesman Problem", 43, pp.73~81 (1997)
- "Explaining temporal differences to create useful concepts for evaluating states", pp.882~888 (1990)
- "The ant system and local search for the traveling salesman problem"
- "ACO Algorithms for the Traveling Salesman Problem" (1999)