Google DeepMind releases Genie 4 world model with real-time audio

In short: Google DeepMind has released a research preview of Genie 4, an AI system that can generate interactive virtual scenes with matching audio and can be steered in real time using text.

What happened

Google DeepMind announced Genie 4, its latest “world model.” A world model is an AI that can create a simulated world you can explore and change, a bit like an improv video game that responds to you.

Genie 4 can produce more realistic looking scenes than earlier versions, according to Google. It also adds real-time audio that fits what is happening on screen, including background sounds, music, and spoken dialogue.

A key change is that users can type new instructions while the scene is already running, and the system will adjust. For example, someone could change the weather, shift the camera angle, or add objects or people without restarting.

Google said Genie 4 can keep a scene consistent for up to two minutes, which is about double the limit of Genie 3. It also ties more closely into Google’s Gemini models, with Gemini helping plan steps while Genie generates the visuals.

For now, Genie 4 is available to “trusted testers” in a research preview. Google plans to roll it out in the coming weeks to Google AI Ultra subscribers through Project Genie, and to early access partners using the Gemini API and Google AI Studio.

Why it matters

Tools like Genie 4 could change how people prototype ideas for games, learning experiences, films, and robots. Still, Google says the system has limits, including details drifting over time, and challenges with making on-screen text readable and audio fully accurate.

Source: TechCrunch AI