Robotic Agents Performing Human-Scale Everyday Manipulation Tasks - In the Knowledge Lies the Power
Enabling robotic service agents to perform natural language instructions such as "flip the pancake" or "push the spatula under the pancake" requires us to equip robots with large amounts of knowledge. To perform such tasks adequately, robots must, for instance, be able to infer the appropriate tool to use, how to grasp it and how to operate it. They must, in particular, not push the whole spatula under the pancake, i.e. they must not interpret instructions literally but rather recover the intended meaning.
In this talk, I will present some of our ongoing research in the European project ROBOHOW, in which we transfer these ideas to autonomous robotics in order to realize knowledge-intensive robot control programs that use the world-wide web as a comprehensive source of knowledge. I will discuss the feasibility of acquiring new plan schemata for web instructions formulated in natural language using video instructions, games with a purpose, and kinesthetic teaching as additional information sources.