SciELO - Scientific Electronic Library Online

 
vol.15 issue3An improved particle filter for sparse environments author indexsubject indexarticles search
Home Pagealphabetic serial listing  

Journal of the Brazilian Computer Society

Print version ISSN 0104-6500

Abstract

SILVA, Valdinei Freire da  and  COSTA, Anna Helena Reali. Compulsory Flow Q-Learning: an RL algorithm for robot navigation based on partial-policy and macro-states. J. Braz. Comp. Soc. [online]. 2009, vol.15, n.3, pp. 65-75. ISSN 0104-6500.  http://dx.doi.org/10.1007/BF03194507.

Reinforcement Learning is carried out on-line, through trial-and-error interactions of the agent with the environment, which can be very time consuming when considering robots. In this paper we contribute a new learning algorithm, CFQ-Learning, which uses macro-states, a low-resolution discretisation of the state space, and a partial-policy to get around obstacles, both of them based on the complexity of the environment structure. The use of macro-states avoids convergence of algorithms, but can accelerate the learning process. In the other hand, partial-policies can guarantee that an agent fulfils its task, even through macro-state. Experiments show that the CFQ-Learning performs a good balance between policy quality and learning rate.

Keywords : machine learning; reinforcement learning; abstraction; partial-policy; macro-states.

        · text in English     · pdf in English