Transparent RLfD

Robot motions in Learning from Demonstration tutorial

• To find out what kind of robot motions will affect future demonstrations

Related robot motion types: attentional motions? uncertainty motions? predicatable motions?

• To know how those robot motions influence the demonstrations (positive or negative?)

Possible influences: how the learning process evolve over time? the state visited, the values updated

Overview

Tentative schedule:

1. Demonstration (Jun 15th)

2. Model-free RL (Jul 1st)

Single state-action update

• Q-Learning

Autonomous Q-learning test on the simulation:

• SARSA

Multiple state-action update: update whole trajectory

• TD(lambda)
• SARSA(lambda)

To be decided:

To-do list:

• State: pose (position & orientation), velocity, physical properties etc.
• Reward: -1 for each movment until reach final goal, -2 for collision, 10 for reach final goal, object pose

Two different algorithms: to compare these two types of algorithms

• Inverse Reinforcement Learning
• • Reference: Showing versus Doing: Teaching by Demonstration
• • Exp 1: Teaching Goal-based Reward function
• • Exp 2: Teaching Feature-based Reward function
• Model-free Reinforcement Learning

3. Learning process evaluation (Jul 15th)

Q-learning:

• Q function update (update for current state and action only)
• Predicted action vs. human input action

4. Motion planning (Jul 25th)

• Testing on PR2 robot: robot right arm follows cup movement: (speed: x10)

• Testing on UR10 robot: (speed: x5)

To-do lists

1. Need to find out the possible robot motions that would affect the human perception
2. Need to implement the related motions
   • Gesturing motions: policy-based

• Pausing motions: state-uncertainty-based

1. Need to determine the learning model and evaluate the model effects towards the learning

Expected outcome

Final outcome

Related source code

Matlab Reinforcement Learning