Face ID: Why you shouldn't be worried about iPhone X unlock

Face ID is Apple's new facial identity scanner. It replaces Touch ID, Apple's fingerprint identity scanner, on the next generation iPhones X.

Following its introduction, here's what Apple told me about Face ID:

Touch ID was a solution to a problem: How to make accessing a secure device faster and more convenient. But it was only one of several potential solutions. Face ID is another. And it's one that has a few downsides but also a significant upside:

Touch ID always had a 1/50,000 chance a random stranger's fingerprint pattern would match yours enough to gain access. For Face ID, that chance drops to 1/1,000,000.

But change is scary and the new is also the unknown. That's why we're already seeing a lot of stress and sensationalism — just like we did with Touch ID.

Let's clear that up.

How does Face ID work?

Face ID works similarly to how Touch ID works but instead of a sensor in the Home button it uses the True Depth camera system on the front of iPhone X.

According to Apple's Face ID Security White Paper, when you first register with Face ID, the True Depth camera system takes infrared images of your face. Just like you had to move your finger around for Touch ID, you have to move your face around for Face ID. That way the camera system can capture you from a variety of angles and create a depth map of your face.

The resulting data is then sent to the secure enclave where a secure portion of the Apple A11 Bionic chipsets Neural Engine Block transforms it into math.

Here's where there's a difference between Touch ID and Face ID: Touch ID threw away the original enrollment images of your fingerprint at this point. Face ID keeps the original enrollment images of your face (but crops them as tightly as possible so as not to store background information). The reason for this is convenience. Apple wants to be able to update the neural network trained for Face ID without you having to re-register your face. This way, if and when the neural networks are updated, the system will automatically retrain them using the images stored in the same region of the secure enclave.

Like Touch ID, that data is only available within the secure enclave, never leaves the device, is never sent to Apple, and is never included in backups or stored on any servers anywhere.

Once you've registered with Face ID, and you go to unlock, here's what happens:

1. Attention detection makes sure your eyes are open and you're actively and deliberately looking at your device. This is to help avoid unintentional unlock. (It can be disabled for accessibility, if desired.)
2. The flood illuminator makes sure there's enough infrared light to "see" your face, even in the dark.
3. The dot projector creates a contrasting matrix of over 30,000 points.
4. To counter both digital and physical spoofing attacks, a device-specific pattern is also projected.
5. The True Depth camera reads the data and captures a randomized sequence of 2D images and depth maps which are then digitally signed and send to the Secure Enclave for comparison. (Randomized to again counter spoofing attacks.)
6. The portion of the Neural Engine inside the Secure Enclave converts the captured data into math and the secure Face ID neural networks compare it with the math from the registered face.
7. If the math matches, a "yes" token is released and you're on your way. If it doesn't, you need to try again, fall back to passcode, or stay locked out of the device.

Update: Making purchases with Apple Pay or through the App Store or iTunes is similar, you simply click the side button twice to cue the system, like you do on Apple Watch already.

The secure neural networks were trained specifically for Face ID resolution using over a billion images, including infrared images and depth maps, that Apple collected during informed studies conducted around the world, with representative groups of people from a wide spectrum of origins and backgrounds.

(Apple has actually deployed more than one neural network — including one trained specifically to defend against spoofing attacks.)

Face ID may also store, for a limited time, the math from successful unlock attempts and even from unsuccessful unlock attempts where you immediately followed up by entering the passcode. That's to help the system keep pace with changes to your face or look that might accrue over time, even the more dramatic ones. After it's used the data to augment a limited number of subsequent unlocks, Face ID discards the data and, potentially, repeats the augmentation cycle.

The result is a robust system that, at least based on my limited observations, works with surprising speed and efficiency.

Is Face ID compatible with Touch ID apps?

Any existing Touch ID app will also work with Face ID. Apple has abstracted away the implementation details and simply lets the app ask for biometric authentication and then, if and when an ID matches, authorizes the app.

For iOS 11, developers can add information specific to Touch ID or Face ID in order to provide a better user experience, but the system itself will work either way.

Developers can also require Face ID for a second factor in secure apps, and generate and use ECC keys inside the Secure Enclave that can be unlocked by Face ID.

Does this mean apps get access to my "face"?

No. Just like apps never got access to your fingerprints with Touch ID, they never get access to your face data with Face ID. Once the app asks for authentication, it hands off to the system, and all it ever gets back is that authentication (or rejection).

Then how do Animoji and face tracking apps work?

Apple has a separate system, built into ARKit, the company's augmented reality framework, that provides basic face tracking for Animoji or any apps that want to provide similar functionality.

All it does is provide rudimentary mesh and depth data, though. It never touches Face ID data or the Face ID process.

Can Face ID require passcode the way Touch ID sometimes does?

Face ID works similarly to Touch ID in that it can lock down and require passcode under certain conditions. Here's Apple's list:

• The device has just been turned on or restarted. >- The device hasn't been unlocked for more than 48 hours. >- The passcode hasn't been used to unlock the device in the last 156 hours (six and a half days) and Face ID has not unlocked the device in the last 4 hours. >- The device has received a remote lock command. >- After five unsuccessful attempts to match a face. >- After initiating power off/Emergency SOS by pressing and holding either volume button and the side button simultaneously for 2 seconds. At any of those points you'd have to use passcode to re-enable Face ID.

Can police or criminals hold your phone up to your face to unlock it without your permission?

One of the biggest areas of fear, uncertainty, and doubt surrounding a potential Face ID facial identity scanner is that it will make it easier to law enforcement and government agencies to gain access to our devices.

(That a significant segment of humanity is more concerned about illegal search and seizure by law enforcement agencies than the criminals they're meant to protect us from should embarrass and appall governments around the world, but that's a different editorial for a different day.)

"Easier" is tough to parse, though. A few years ago, when Touch ID was introduced, we saw similar concerns: That someone could wait for you to fall asleep, or incapacitate or restrain you, and then simply touch your finger to the sensor to unlock your phone. if they didn't know which finger(s) (or, humorously, other body part) you'd used for for Touch ID, there was a chance they could trigger Passcode lockdown after repeated failed attempts, then simply torture you for the answer and/or passcode, but the tendency towards convenience made the thumb the likely digit in almost all cases.

What happens in the real world can always be different than what is later ruled admissible or inadmissible in the courts, and illegal search and seized didn't begin or end with Touch ID and won't begin or end with Face ID. Likewise for individuals on the criminal side who want access.

Face ID, by default, requires you to be looking at iPhone X to unlock. So if you close your eyes and look away, someone would have to force you to open your eyes again for Face ID to unlock. Just like they'd have to force your thumb onto Touch ID to unlock. It would also make it very difficult to have Face ID unlock if you're sleeping, which is currently much easier with Touch ID.

For people who require security, a strong password is available. For those who want convenience, biometrics. Don't want to risk your finger or face being used against you, don't use your finger or face for unlock. (Or disable it by quintuple-clicking the Side button before going places or entering situations you don't trust.)

In a perfect world, Apple would let those who want even more security to require both password and biometrics — and a trusted object like Apple Watch as well. Hopefully, Apple is working on that.

What about masks or makeup, can they fool Face ID?

Another source of concern has been the use of photos, videos, makeup, masks, and evil twins — perhaps even and up to plastic surgery — to gain access through the Face ID system.

Part of the reason for this is too much Hollywood. The other part is poor implementations of face scanning to date, similar to how we had poor implementations of fingerprint scanning before Touch ID.

Yet Hollywood is who Apple turned to to help train the Face ID system. Photos were never a major concern because of how Apple scans faces in 3D space. But, Apple had practical effects artists create makeup and masks to try to fool Face ID, then used that makeup and those masks to train the neural network on the Apple A11 Bionic chip to prevent against just that type of attack.

For example, if an attacker took all the social media photos of you they could find and used it to build and print and 3D mask.

Video spoofing attempts, which can be much harder to detect than static, single-dimensional photos, was similarly trained against.

Will Face ID work in the dark? What about at angles?

People have also been concerned about the utility of Face ID at night or in the dark, at angles like when the phone is flat on a table, and when in use, like for authorizing Apple Pay.

For no or low light, Apple is using a "flood illuminator" to make sure your face remains scannable to infrared. Then the "dot projector" hits you with 30,000 points for identification purposes. The camera system then captures the data, converts it into math, and pushes it through the Neural Network block on the A11 Bionic chip to see if what was captured matches what was registered. If it does, you're authenticated.

The camera system has a fairly wide field of vision so, depending on the exact angles, you can tap the screen to wake iPhone X, and be scanned by Face ID, under a variety of conditions.

You can also disable "attention", which requires you to be looking directly at iPhone X for Face ID to unlock, but for most people leaving it on is better for security.

Can Face ID work through scarves, beards, and glasses?

Because Face ID uses the Neural Network block in Apple's A11 Bionic system-on-a-chip, it continually learns and adapts to how your face, facial hair, hairstyle, and facial coverings vary and change over time.

With the "dot projector" making up to 30,000 points available, there's a lot of data for Face ID to work with. Apple also built and tested the system to make sure it works under the widest varieties of conditions possible.

That includes if you have your glasses on or off (though sun glasses might obscure "attention" mode, which is on by default). If you change your hair style or color. If you grow a mustache and/or beard and then shave them off. If you're wearing religious or climate-based facial coverings (though full-on Canadian-style ski masks and goggles might obscure too many points of identification to be useful — I look forward to testing that!)

There may be cases where too much changes at once and Face ID fails to recognize you. If/when that happens, the system will simply kick you back to Passcode, just like Touch ID does when the moisture level (or wetness) of your finger was different enough to prevent a scan.

How about Apple Pay, isn't that weird with Face ID?

For Apple Pay, the process is largely unchanged.

Double-clicking the Side button replaces double clicking the Home button to invoke Apple Pay, glancing at Face ID replaces holding on Touch ID, and the tap on the NFC payment terminal remains the same.

If you typically invoke Apple Pay buy tapping the NFC terminal first, you may have to pull back to authorize with Face ID, depending on the exact positioning.

Does Face ID work for those with low or no vision? How's the accessibility?

For people who have low or no vision, Face ID will guide you through the setup and authentication process, including cueing you on positioning.

You can also disable "attention" so, even if you're not looking directly at iPhone X, it will still unlock when enough of your face is within the system's field of vision.

Are there any limitations to Face ID?

Yes, there absolutely are some limitations with Face ID that could affect people.

• If you're under the age of 13, your facial features may not yet be distinct enough for Face ID to function properly and you'll have to revert to passcode.
• Face ID can't effectively distinguish between identical twins (or triplets, etc.) If you have an identical sibling and you want to keep them out of your iPhone X, you'll have to revert to passcode.
• Unlike Touch ID which allows for the registration of up to 5 fingers, Face ID currently only allows for one face. That means no sharing easy access with family members, friends, or colleagues.
• Face ID does need to have your face within view of the camera system to unlock. Touch ID can work regardless of the camera system orientation. That means, in some situations, you will have to adjust your position for Face ID to work.
• If, for any reason, you don't like the idea of your face being scanned, you'll have to revert to passcode.

Just like Touch ID improved over time, Face ID should improve. For now, though, those may be your showstoppers.

Any Face ID questions?

It's important to remember that biometrics are a convenience. Before Touch ID, many people wouldn't even go to the trouble of using a 4-digit passcode to lock their iPhone. Now, thanks to the convenience, many of them do. It's nowhere nearly as secure as using a long, strong, pseudorandom password for every unlock, but most people won't ever be willing to go through that process daily, much less hundreds of times a day. So, biometrics.

No doubt we'll go through all the same stress and sensationalism with Face ID we went through when Apple introduced Touch ID back in 2013.

We'll see people with CSI-level resources making dummy heads to try to fool it — and it's quite possible some will succeed in the same way they succeeded with Touch ID. We'll see headlines about how your face can now betray you, and goofy examples of facial contortions recommended to avoid forced scanning. We'll see... everything we typically see whenever Apple introduces any new feature.

And then we'll use it, forget the fuss, and move on. Just like we did with Touch ID. Just like we do every year.

Updated September 27, 2017 with information from Apple's Face ID white paper.

Updated September 15, 2017 with comment from Apple.

Updated September 13, 2017 with information from Apple's iPhone X announcement event.