Discover Relevant Environment Feature Using Concurrent Reinforcement Learning

Zhihui Luo, David Bell, Barry McCollum

When a robot observes its environment, there are two im-portant characteristics of the perceived information. One is the relevance of information and the other is redundancy. The irrelevant and redundant features which commonly exist within an environment, commonly leads to state ex-plosion and associated high computational cost within the robot’s learning process. We present a method concerning the relevance of infor-mation in order to improve the learning of a reinforcement learning robot. We introduce a new concurrent online learning algorithm to calculate the contribution C(s) and relevance degree I(s) to quantify the relevancy of features with respect to a desired learning task. Our analysis shows that the correlation relationship of the environment features can be extracted and projected to concurrent learning threads. By comparing the contribution of these learning threads, we can evaluate the relevance degree of a feature when performing a particular learning task. We demonstrate the method on the chase object domain. Our validation results show that, using the concurrent learning method, we can efficiently detect relevant features from the environment on sequential tasks, and therefore improve the learning speed.

Subjects: 12.1 Reinforcement Learning; 17. Robotics

Submitted: Apr 1, 2008
