MIT Research Shows LLMs can Help Home Robots Recover From Errors Alone

MIT research revealed that large language models (LLMs) can help home robots develop a bit of "common sense" and learn how to recover from mistakes.

Although robots are excellent at mimicking commands, issues can exhaust the pre-programmed options which forces human intervention to solve the problem.

LLMs to Help Mitigate Human Intervention From Robotic Errors

According to the researchers, exhausting the pre-programmed options is a common problem among robots in an unstructured environment like a home. During its first try, the robot will create a virtual map, noting every nuance on its path to be avoided, however, problems can still occur.

Despite home robots' impressive skills, they don't often account for the countless small environmental variations that could disrupt their regular operations. Sometimes the robot would have to restart from square one again.

The usage of LLMs as a training tool will hopefully help the robots to know what actions to take, just like how a human would function in case of any disturbance.

"And we wanted to connect the two so that a robot would automatically know what stage it is in a task, and be able to replan and recover on its own," said grad student Tsun-Hsuan Wang.

LLMs to Train Robots Through Series of Subtasks

The study had a demonstration by training a robot to scoop marbles and pour them into an empty bowl. Although it seems like a very simple task for humans, robots consider this as a series of small tasks.

Through LLMs, the robot could process it by listing and labeling them. During the experiment, researchers would sabotage the activities to see how the system would respond on its own instead of starting from scratch.

"With our method, when the robot is making mistakes, we don't need to ask humans to program or give extra demonstrations of how to recover from failures," Wang added.

The study will be presented at the International Conference on Learning Representations in May.