Programming intelligent household robots using a virtual research environment

Michaela Kümpel

Have you always wondered what is needed to program a robot to bring milk to the coffee table for breakfast? If so, you can find out in this project-framed course! The course is designed to teach the principles needed for developing a robot application that does just that: bring milk to the coffee table, in simulation. Using a virtual research environment running on a website that does not require software installation or specific hardware, we will go over four main aspects needed to create a robot application: 1) Modelling the environment. A robot uses its sensors to make sense of its surroundings, but the sensor data needs to be labeled so that some object can be detected as a table, or a fridge. We will create a kitchen for the robot to act in. 2) Now we need a robot in our kitchen. The robot needs to be told to look for objects and match this to the environment model. We will integrate a robot model and perform a basic perception task. 3) With a robot in a kitchen, the next thing needed for performing the task is to give the robot knowledge about what a milk is and where a milk can be found. We will discuss knowledge engineering and let the robot infer where the milk would be stored. 4) Finally, the robot is equipped to perform the task of bringing milk to the table. Now we need to write an action plan for the robot so that it knows what to/ how to move to perform the task. We will instruct the robot to move to the fridge, grasp the milk, and bring it to the table.

During the program, we will also address problems that arise when robots operate in human environments. For example, the importance of creating a trustworthy interaction, and how to achieve this, but also how to include preferences of humans if one prefers lactose-free milk, or soy milk. We will further ask ourselves if different age groups, genders or people with different abilities would need robots to act in different ways, and how.