Robots that Learn to Score Goals

Soccer Robots Learn to Shoot Goals

This work, originally done in 1997 by Rahul Jain and Giridhar Rajaram, learns how to shoot a goal by a single robot with on-board camera. In the same year, IIT Kanpur Center for Robotics participated as part of a joint international team at the Micro-Robot Soccer tournament in Korea. Micro-Robot soccer trades-off the full autonomy of on-board camera for the very small size of 70mm robots - a single camera is mounted above the board and tracks all the players and the ball. Unlike this robot which learns how to shoot into an empty goal, current robots are able to learn to shoot goals with goalie and other opponents, as well as to send and receive passes. By combining the maximal probability of success of either of these two tasks, the robot agent can decide whether to shoot or to pass.

STRIKER- an appropriate name for a soccer playing robot is a visually operated soccer playing robot, one of the first of its kind in the country.

Reinforcement Learning in Visual Space

Earlier approaches have tended to see vision-based soccer playing robots as one unified system with a global task of shooting goals. This results in learning taking longer to converge, as well as in lower success rate. We have viewed a soccer-playing robot as a reactive-deliberative system. The global task of shooting goals was decomposed into three subtasks that formed a hierarchy:

(i) Find ball,
(ii) Align with the ball and goal, and
(iii) Kick the Ball

- with (iii) being the top layer of this hierarchical control structure. Only when the lower layer behaviours are satisfied, does a higher layer behaviour get activated. While the reactive behaviours are learnt, the deliberative behaviours are based on heuristic (no learning). A reinforcement learning scheme ( Q-Learning) was used on each of the reactive type of subtasks. Learning from Easy Missions (LEM) mechanism was used to further improve the learning rate. A higher success rate results from task decomposition, as each of the subtasks is a much more tangible task than the global task of scoring a goal without any preliminary notion of how to do so.

The image processing is done using thresholding and texture recognition. A CCD camera is on board the robot which captures frames. The goal is made out of black and white squares (which gives the impression of a net) and a white ball is used. The image processing is quite effective and a good threshold helps in segregating the environment out of the captured frame.

These are figures of Striker while scoring a goal:

For further information, contact:
Amitabha Mukerjee at amit@iitk.ac.in or Giridhar Rajaram at graj@rocketmail.com or Rahul Jain at rahulj@hotmail.com