AR and ARKit 2: Everything you need to know!

With the update to iOS 11 last fall, Apple created the single biggest augmented reality platform on the planet. Millions of people suddenly had access to a massive catalog of apps which seemed to leap off your phone and onto the real world around you. These features were made possible through a collection of developer tools Apple called ARKit, and the company has made regular updates to the software to improve it dramatically since its initial announcement at WWDC last year.

With iOS 12 on the way, Apple revealed the next steps for ARKit, growing it from version 1.5 to version 2.0 in this new release. Here's what you can expect from the next wave of ARKit apps and experiences thanks to ARKit 2!

What's new in ARKit 2

Where ARKit 1 and ARKit 1.5 focused on tools for developers to locate the physical tracking space for users to interact with, ARKit 2 is about being able to keep an AR space around after you have created it. In your average AR experience today, after you close the app it all goes away. If you were in the middle of a puzzle, for example, you have to build a map of a new place and pick a new spot for that puzzle to live in order to go back to that experience. New tools in ARKit 2 make this a lot less work with Persistent Experiences, which make it so you can come back to a room and all of your AR things will still be there to enjoy.

These tools also create a "world map" and allow users to share that space with others. When another user joins an ARKit world, each can see the same augmented creations from their own devices. People can play ARKit games together, schools can use ARKit for a shared educational experience, and so on. The important thing is being able to quickly grab a map of the room you are in and be able to share it with another iPhone or iPad quickly.

A big part of what will make all of this work quickly and effectively, no matter what app you are using, is the use of a new universal file format called USDZ. This file format was originally developed by Pixar, and Apple has made it a standard for ARKit files on iOS. With support from Adobe and several other companies, creators will be able to build new ARKit images with this file format and embed them in everything from apps to websites. It also means Apple can influence WebAR experiences, which had not previously been the case.

How do Shared ARKit Experiences work?

Apple's documentation for what it calls Shared ARKit Experiences looks like a multi-step set up for whatever app supports it, but that process should be fairly simple and easy to do. Essentially, the first user opens the AR app they want to play and lets the app create a world map. Once that map is complete, a second user can open the app and try to join that first user.

When the apps are aware of multiple users in the same space, a Send World Map button will appear. That will allow you to send your world map to another user, making sure you both have the same area to explore together. Once both apps have the same world map, the Shared ARKit Experience can begin.

On a technical level, this is very similar to Google's Cloud Anchors system for ARKit and ARCore. Unlike Cloud Anchors, Shared ARKit Experiences is only designed for iOS-based hardware to share. If the same app is on an Android phone, the experience will not be shared.

Which phones can use ARKit 2?

Like the last update to iOS 11, all you really need to take advantage of ARKit 2 is an iPhone or iPad with iOS 12 onboard. We already know iOS 12 is going to be one of the most widely supported iOS releases in its history, so if you are using an iPhone or iPad right now there's a good chance you will have access ARKit 2 and all of the cool things it comes with.

That having been said, most iPhones and iPads only have access to ARKit experiences through the rear camera. If you want ARKit experiences through your front camera, with things like Animoji and the new Memoji, you need to have a phone with a TrueDepth sensor on the front. That means you need an iPhone X for now, with more phones likely supported later this year.

What is Augmented Reality?

Where VR replaces everything you see by completely immersing you in another world, AR takes the world around you and adds to it. For Apple, this means looking through the camera on your phone and seeing new things in addition to the real world.

The applications for this kind of technology are nearly endless. You could point your camera at a painting and have details about the artist appear nearby, or see how furniture would look in your office before buying and moving it. And, of course, AR opens the door to a lot of interesting and fun games.

What does Apple want to do with AR?

Basically, everything. Apple is betting on developers wanting to build AR experiences for the millions of iOS users out there and offered plenty of demos during its WWDC keynote and demo areas to prove it. Through ARKit, Apple's new toolkit for AR apps, we've seen demos where the table in front of us has been completely overrun by something virtual and interactive.

For Apple, this is an entirely new category of apps that would all run on every iPhone. Long-term, it's possible Apple has plans to build special displays you wear like prescription glasses that make the Augmented world a part of your everyday experience.

Check out our full review of ARKit for iOS

Where did ARKit come from?

Apple doesn't usually talk about the origins of its technology and frameworks, but an educated guess would be Mataio, and AR company Apple aquired a couple of years ago.

Mataio's technology looked solid and it had an API that, according to reports, was already very good.

How does this compare to the other stuff out there right now?

ARKit's apps are undeniably impressive and fun, but clearly limited when compared to Microsoft's Hololens or Google's Tango AR kits. Hololens and Tango are designed to be aware of multiple physical spaces and all of the shapes contained within, like a glass on a table or the distance between where you are standing and the wall on the other side of the room. ARKit as it exists right now is able to find flat surfaces and draw on those flat surfaces, but is otherwise limited.

The best thing to compare ARKit to right now is the Facebook Camera app. ARKit is a more capable version of the AR capabilities contained within Facebook's app right now, with the potential to continue improving as we move closer to the launch of iOS 11. Apple is able to use the whole phone, which means systems like Unity and Unreal Engine will allow for much more dynamic and interactive experiences once developers have had time to explore.

How do I get started with ARKit?

The only think you need to do to get started with ARKit is open any app using this technology. This is meant to be a largely invisible force full of new ways to use your phone. Apple views ARKit as something to be deeply integrated into your phone, eventually becoming a part of your phone you can't imagine not having. The best example of this is the iPhone camera, specifically the Portrait Lighting and Animoji features. There are also healthy number of apps which are entirely Augmented Reality, and a lot of them end up being a lot of fun games or social media additions.

Check out our favorite ARKit apps!