Apple Vision Pro will identify items, text, and more before your very eyes

(Image credit: Apple)

Following the release of visionOS and an accompanying SDK to developers yesterday, we're starting to learn more and more about what the Vision Pro headset will be capable of when it ships in early 2024. And the things that we're learning sound pretty great.

Apple announced Vision Pro during the WWDC 2023 opening keynote earlier this month but it didn't have time to show everything off. Thankfully, developers picking through visionOS have already started to share details about what the headset will do, including support for a new Visual Search feature.

From what we're told, Visual Search will work in a similar way to the already impressive Visual Lookup — a feature that can look at the content of a photo and then provide information and context on it. Like identifying in-car symbols, for example. And that means that Vision Pro will be able to identify objects in the real world and provide information on them right in front of your eyes.

Basically, magic

Visual Search was spotted inside the visionOS software by developer Steve Moser and reported on by MacRumors. Moser later confirmed that Visual Search appears to be part of Spotlight rather than the Vision Pro's camera system which might be the opposite of what some would have expected.

It looks like 'Visual Search' is part of Spotlight instead of the camera. I wish Apple had shown off this feature and ARKit during the keynote. Related accessibility feature: ""Describe Passthrough" requires Image Captions to be enabled in Settings Accessibility > VoiceOver." https://t.co/e0GPqjvECcJune 22, 2023

However it's invoked, Visual Search has the ability to identify items, detect text, and even manipulate that text for use elsewhere. It's also expected that Vision Pro will offer to translate text if it's in a foreign language with as many as 17 supported out of the gate. We can also likely expect the headset to do unit conversions as well.

We might learn more about this feature as we get closer to the release of the Vision Pro headset. That isn't expected to happen until early 2024, unfortunately. The $3,4999 asking price is likely to put a few people off, however.

Looking to save some money ahead of the big release? Be sure to check out our collection of the best Prime Day Apple deals before making a purchase.

Oliver Haslam has written about Apple and the wider technology business for more than a decade with bylines on How-To Geek, PC Mag, iDownloadBlog, and many more. He has also been published in print for Macworld, including cover stories. At iMore, Oliver is involved in daily news coverage and, not being short of opinions, has been known to 'explain' those thoughts in more detail, too. Having grown up using PCs and spending far too much money on graphics card and flashy RAM, Oliver switched to the Mac with a G5 iMac and hasn't looked back. Since then he's seen the growth of the smartphone world, backed by iPhone, and new product categories come and go. Current expertise includes iOS, macOS, streaming services, and pretty much anything that has a battery or plugs into a wall. Oliver also covers mobile gaming for iMore, with Apple Arcade a particular focus. He's been gaming since the Atari 2600 days and still struggles to comprehend the fact he can play console quality titles on his pocket computer.