AI firms push "world models" to train robots and driverless cars

In short: AI companies are increasingly building “world models,” which are AI systems that learn from images and video to simulate real places and how objects move.

What's going on

World models are a shift from AI that mainly works with text to AI that can “understand” spaces. Instead of predicting the next word in a sentence, a world model can look at a scene and predict what it will look like next. Think of it like a very advanced video prediction system that also tries to follow real-world rules like gravity and collisions.

Some of the raw material for these models comes from everyday people. Wired points to Niantic, the company behind Pokémon Go, which used player phone scans of neighborhoods. Niantic has since pivoted toward building 3D copies of the real world that could help robots navigate and interact with objects.

Startups focused on cars are doing something similar. Wayve, for example, uses dashcam footage to recreate dangerous road situations that would be risky to test in real life. Game companies are also paying attention, since video games already rely on 3D worlds and physics.

Big funding and deals are following. Wired notes that Yann LeCun raised over $1 billion for a startup aimed at AI that understands the physical world. It also reports that Fei-Fei Li’s startup World Labs was acquired by AMD for over $8 billion.

What to watch

As world models improve, more robots and driverless car systems may be trained in simulations first (like pilots using flight simulators before flying a real plane). Another thing to watch is the growing demand for real-world video, including gig work where people record chores or daily activities to help train these systems.

Source: Wired