Apple warned Grok app could be removed over sexual deepfakes

In short: Apple privately warned Elon Musk’s Grok app it could be removed from the App Store unless it better stopped nonconsensual sexual deepfakes.

What happened

Apple contacted the teams behind X and its AI chatbot app, Grok, after complaints and news reports about sexual deepfakes, according to a letter seen by NBC News and reported by The Verge. A deepfake is a fake image or video made with AI that can make it look like a real person did something they did not do.

Apple told US senators it asked the developers to create a plan to improve “content moderation,” meaning the rules and systems that try to stop harmful posts (like a bouncer checking who gets in). Apple said X had “substantially resolved its violations,” but Grok “remained out of compliance” at first.

Apple said it warned Grok’s developer that more changes were needed or the app could be removed from the App Store. After more back and forth, Apple said Grok had “substantially improved,” and Apple approved an updated version.

The Verge reports that during this period, Grok and X appeared to stay available in the App Store. Changes included limiting Grok on X to paying subscribers and adding tools that let people block Grok from editing their photos, though The Verge says these steps were easy to work around.

Why it matters

Apple and Google act like gatekeepers for phone apps, and removal from their stores can cut off a major way people download a product. This case also shows how hard it still is to stop AI tools from being used to make sexual images of real people without consent, even after promised safeguards.

Source: The Verge AI