Google adds Guided Vision to Gemini Live for real-time camera help

In short: Google is launching Guided Vision in Gemini Live on compatible Android devices to give spoken, real-time descriptions of what your phone camera is pointed at.

What happened

Google is rolling out a new feature called Guided Vision inside Gemini Live. Gemini is Google’s AI assistant, and Gemini Live is its mode for back-and-forth voice conversation.

With Guided Vision, a person can share their phone’s camera view with Gemini Live and hear audio descriptions of what is nearby. Google says it can help with tasks like reading small text, describing your surroundings, finding an object, identifying an object, and describing details on a specific item.

The feature also supports follow-up questions. For example, someone could ask Gemini to read an expiration date after it helps locate the right item. Google says Gemini can give audio cues to help you aim the camera if what you asked about is not currently in the shot, like a friend saying “a little to the left.”

Google says Guided Vision is meant for people who are blind or have low vision, and also for anyone who wants extra help in certain situations. In addition to the Gemini app, it is available through TalkBack, which is Android’s built-in screen reader for accessibility. Users can also set it as an accessibility shortcut in Android settings on devices running Android 9 and above.

Google also warned that Guided Vision should not be used for navigation, safe-travel guidance, or obstacle detection. It is not meant to replace a cane or other mobility aid.

Why it matters

For many people, everyday tasks like reading fine print on labels or confirming details on packaging can be difficult. Features like Guided Vision aim to make a phone act more like a helpful guide that explains what it sees, while still requiring caution for anything related to physical safety.

Source: The Verge AI