Scaling Up Robotics Datasets

In this seminar, I discussed how to make robots useful in unstructured environments outside factories. Robotics has made considerable progress, but much of the work still focuses on individual skills. How can we build robots that perform a wider variety of tasks?

I looked at lessons from computer vision and natural language processing, including large language models and multimodal systems. Network structure, inductive bias, flexibility, expressiveness, and access to large amounts of data all matter for robot learning.

I also compared real-world experiments with simulation. Simulators offer parallelization, flexibility, and access to a wider range of data, but a gap remains between simulated and real-world settings. One idea I discussed was whether sufficiently capable simulators could treat real-world scenarios as a special case.

The talk argued for using both simulation and real-world testing to develop robots with more varied and useful capabilities.