Google adds Gemini Intelligence features that can act inside Android apps

In short: Google previewed “Gemini Intelligence,” a bundle of Gemini features for Android that can help control apps, fill forms, and build custom widgets.

What happened

Google announced new Gemini features during its Android showcase ahead of Google I, O. The company is grouping some of these features under a new name, Gemini Intelligence. Google says it is meant for its most advanced Android devices, and it may be limited to premium phones at first, such as newer Samsung Galaxy and Google Pixel models.

One focus is “task automation,” which lets Gemini carry out actions in certain apps for you, like a helper that taps buttons on your behalf. Google says this will expand beyond a small set of ride share and food delivery apps to more apps “soon.” Gemini will also accept screenshots and photos as part of these requests, for example, using a screenshot of a grocery list to add items to a shopping cart.

Google also showed “Create My Widget,” which lets you describe a widget (a small information box on your home screen) and have Gemini create it. Examples include a cycling weather widget that highlights wind and rain, or a recipe widget that suggests specific meal plans. Google says these widgets will also work on Wear OS watches.

Gemini is also coming to Chrome on Android, with a button to ask questions about a web page. For subscribers to Google’s AI Pro or Ultra plans, Google says “auto browse” will help complete tasks like booking appointments, starting in late June.

Gemini can also be connected to Android autofill, optionally. Google says this could pull details from services like Gmail or Google Photos to help complete forms.

Why it matters

These features could save time on everyday phone tasks, but they also raise new privacy questions since autofill may use more personal data. Google says features will roll out in waves through the year, with Pixel and Galaxy phones first starting this summer.

Source: The Verge AI