Wednesday, 24 June 2009

AIR 2009 day 1

For this week (23rd - 30th June) I will be at the Autonomous Intelligent Robots Summer Camp. This is at the University of Wales, Aberystwyth. The website has lots of details about the specifics, including a timetable of stuff that's going on.

This morning was a talk by Jeremy Wyatt about reinforcement learning which is when you use rewards and punishments to make a robot learn about its environment and how to react to it and make predictions about the outcome of its actions. I didn't know very much about it before, I'd read something about game theory in draughts from one of John Holland's books which sounded very similar. Basically a robot can observe the state of the world, and it has a policy which control how it acts when given a particular world state. The robot gets given rewards based on what you want it to do, like if you want it to push a box you can reward it whenever it pushes something. The robot has to modify its policy in order to maximise the amount of rewards it gets for ALL world states. There are associated issues like it could come up with a behaviour you don't want in order to get lots of rewards (like the box pushing robot might try to push a wall) so you need to be careful when designing the reward system. Also if rewards are too infrequent then the robot can never really learn because it is not told when it does good. He briefly mentioned something called inverse reinforcement learning, which is when you develop a reward system somehow and then use that to teach a robot, then what you teach the robot is fed back into the reward system and modifies it based on the learning process. This is then iterated and so you get a good reward system and a good robot!

This was followed by some lectures by members of Aberystwyth's robotics department who were talking about the things that are going on in their lab.
The first of these was by Mark Lee who talked about developmental robotics. Which is all about teaching robots to do a task in stages, much like how children/babies learn to do stuff gradually. They have a robot arm which has developed hand-eye coordination, they started by developing an eye and teaching it to centre upon certain coordinates so that it can move quickly towards a point instead of circling around it or searching or whatever. They then taught the arm to understand how its motor movements and positions map to its position on a table (looking down) which uses the eye to look at the robot's actual position. Then they taught the robot how to move to coordinates or objects on the table, effectively hand-eye coordination, since the robot arm know what position it needs to be in to get to a place and it can make minor modifications to its position given what it observes with the eye. Good stuff! It makes sense because they don't try to run before they can walk. Often, problems are encountered because people try to do too much at once and it's good to try and increment the difficulty of a task. Humans develop hand-eye coordination in a similar manner and we've been doing that for thousands of years, so the system must be good!

Another talk was on using a 360 degree panoramic camera to navigate to a certain point. This assumes you have a starting picture of where you want to be, they then find the difference between the current camera's image and the one of the goal location. The place where the difference is least will be where you want to go, but as you near that place the difference gets smaller and smaller so it allows you to home in. The good thing about this is that it works better if you use it in a visually rich environment (ie there are lots of colours and blocks and stuff) because it's basically landmark navigation. This is cool because a lot of vision analysis stuff for robots involves putting the robots in an arena where there's not much to look at so interesting things are easy to pick out, this work is much more applicable to real life because the real world isn't painted white!

The final presentation was about the robot scientist they have in Aberystwyth. This has an understanding of yeast biology and uses that to perform and analyse tests on yeast. It's basically a way of automating science so that you can do a lot of tests in a repeatable manner. They use it for identifying what some yeast genes are responsible for and it's been quite successful in that regard! It can apparently run ~1000 experiments a day and can make over 200,000 observations. Good stuff! We went to see it acutally, and it looks very cool.

More fun stuff will happen tomorrow I hope, now I'm off for dinner with everyone.

No comments:

Post a Comment