At its I/O 2024 conference on May 14, Google announced a swathe of new features and tools for users across its entire ecosystem, including Android, Google Workspace, and beyond. As you might expect, the event was laser-focused on the power of AI, and how Google’s Gemini (as well as a handful of other exciting tools) is helping to revolutionize how we interact with smartphones, the web, and the world around us.
The announcements fall mere weeks ahead of Apple’s own WWDC 2024 keynote, where the company is expected to unveil its new iOS 18 operating system for iPhone, as well as iPadOS 18, macOS 15, and watchOS 11. Just like Google, Apple is expected to go all in on AI. Apple AI, however, is not expected to be a major, distinctive product, but rather a series of under-the-hood changes and tweaks to various apps and features. Previous reports claim Apple will focus on tools that help manage your daily life, rather than ChatGPT-like generative AI features.
Indeed, Google’s own announcements this week came with several such quality-of-life improvements we’d very much like to see Apple leverage one day. Here are five announcements from Google I/O 2024 that Apple should borrow for its own products.
Scam call monitoring powered by AI
Android's new scam call monitoring feature caught my eye the most at I/O 2024. “We’re testing a new feature that uses Gemini Nano to provide real-time alerts during a call if it detects conversation patterns commonly associated with scams,” Google says. The company claims that Gemini Nano (its lightweight large language model) can listen to calls live as they happen, detecting those that might be scams.
For instance, Gemini Nano might alert you if someone calls asking you to make a bank transfer or hand over personal information like a PIN or password. The monitoring is done on-device, so there shouldn’t be any privacy qualms, but we’ll know for sure when the opt-in feature goes live later this year. If it works, it could protect vulnerable users from scam calls, something iPhone users could also benefit handily from.
AI-powered accessibility - TalkBack x Gemini Nano
Apple has often led the industry with smartphone accessibility, but Android will soon boast a feature that the iPhone can’t match (at least not yet, anyway). Gemini Nano will soon offer support for clearer and richer image descriptions, essentially live alt-text narrated by AI.
This will provide a spoken description of images for vision-impaired and blind users, the same way a screenreader might be able to read out alt-text currently. As demonstrated by Google, the feature can be used beyond the web (where you’d expect to find alt-text) in apps like Messages, providing AI-powered alt-text for images shared between family and friends.
Circle to Search and AI-powered math
While there is a quick hack that lets you use Android’s Circle to Search feature on iPhone using Shortcuts, the native Android version is much more intuitive and now boasts new AI tools. And at Google I/O 2024, Google said Circle to Search can now help students with their homework: When they circle a prompt, such as a math equation, they’ll get step-by-step help that crucially, won’t do the work for them.
Ask Photos
While Apple’s Photos app for iPhone has handy tools to create fetching albums and memories of your dog, parties, and more, Google Photos features a new Ask Photos option powered by Gemini. Currently, in its experimental phase, Ask Photos can use questions to sift through your massive photo library to draw specific images based on prompts.
The example Google uses is “Show me the best photo from each national park I’ve visited.” Photos are presented along with a textual summary of the photos displayed. Ask Photos isn’t just useful for finding photos, but also for extracting information from them using both the context and the subject of the photos. Google demonstrates: “For example, you can ask: 'What themes have we had for Lena’s birthday parties?' Ask Photos will understand details, like what decorations are in the background or on the birthday cake, to give you the answer.”
Gemini on top of your app
Our final pick is less a specific feature and more an application of AI that Apple’s best iPhones, and specifically Siri, could really benefit from. Largely, using Siri on an iPhone in its current state interrupts whatever it is you happen to be doing. While Siri no longer takes up the whole screen when you activate it, Google is taking Gemini one step further by adding Gemini on top of the app you’re already using.
Users will soon be able to drag and drop generated messages from Gemini into message conversations or use Gemini to search for information in a video they might be watching. The feature will also extend to Google’s Gemini Advanced (its spicier, more capable AI) product, which lets you “Ask this PDF”, scouring massive documents for key information.
Now, it's Apple's turn
These are five interesting, AI-powered features that feel much more in line with Apple’s rumored AI approach. Namely, small, on-device tools that will integrate seamlessly with existing workflows, apps, and features, rather than massive new generative AI tools. With Google’s I/O 2024 out of the way, the ball is now firmly in Apple’s court, and the company has its work cut out to convince users who want their AI fix that they shouldn’t look elsewhere.
