News Archives

[Colloquium] TEXPLORE: A Reinforcement Learning Algorithm for Robots

March 5, 2013

Watch Colloquium: 

M4V file (678 MB)

  • Date: Tuesday, March 5, 2013 
  • Time: 11:00 am — 11:50 am 
  • Place: Mechanical Engineering 218

Todd Hester
Post-doctoral researcher and research educator
Department of Computer Science
University of Texas at Austin 

Robots have the potential to solve many problems in society, because of their ability to work in dangerous places doing necessary jobs that no one wants or is able to do. One barrier to their widespread deployment is that they are mainly limited to tasks where it is possible to hand-program behaviors for every situation that may be encountered. For robots to meet their potential, they need methods that enable them to learn and adapt to novel situations that they were not programmed for. Reinforcement learning (RL) is a paradigm for learning sequential decision making processes and could solve the problems of learning and adaptation on robots.

While there has been considerable research on RL, there has been relatively little research on applying it to practical problems such as controlling robots. In particular, for an RL algorithm to be applicable to such problems, it must address the following four challenges: 1) learn in very few actions; 2) learn in domains with continuous state features; 3) handle sensor and/or actuator delays; and 4) continually select actions in real time. In this talk, I will present the TEXPLORE algorithm, which is the first algorithm to address all four of these challenges. I will present results showing the ability of the algorithm to learn to drive an autonomous vehicle at various speeds. In addition, I will present my vision for developing more useful robots through the use of machine learning.

 

Bio: Todd Hester is a post-doctoral researcher and research educator in the Department of Computer Science at the University of Texas at Austin. He completed his Ph.D. at UT Austin in December 2012 under the supervision of Professor Peter Stone. His research is focused on developing new reinforcement learning methods that enable robots to learn and improve their performance while performing tasks. Todd instructs an undergraduate course that introduces freshmen to research on autonomous intelligent robots. He has been one of the leaders of UT Austin’s RoboCup team, UT Austin Villa, which won the international robot soccer championship from a field of 25 teams in 2012.