The cool thing about the virtual reality conversations at GDC this year is how focused everyone seems to be on improving what we have now. Developers are having great conversations about what they have done wrong so far, and sharing best practices to ensure everyone's next project has a better chance of survival. While these conversations are happening in the convention center, hardware people are off in different rooms showing people what features the next piece of hardware will have behind closed doors.
The big thing coming to VR headsets within the next year is eye tracking, and on a high level everyone should be very excited. Eye tracking is going to allow instant authentication into your collection of apps and services, but also improve every experience you have in VR right now. Your eyes will appear more realistic and human in social VR apps, puzzles will become faster and more dynamic, the world of interactive experiences is truly about to open up and some fun things are going to come out.
But it's also important to remember this is being accomplished by giving some company somewhere unlimited and potentially very detailed access to your eyeballs. It doesn't get much more personal than that, and it is important for every link in the chain responsible for delivering your eyes to those sensors to be involved in keeping this personal information safe.
It's not hard to make eye tracking in this context sound deeply creepy, and quickly turn users off to this kind of feature.
We think about eye tracking in a couple of different ways right now. Some phone manufacturers are starting to use the front-facing camera and an infrared sensor to scan your irises as a way to unlock your phone. Apple's Face ID tech can be used for dynamic eye tracking for some trippy visual effects in coordination with ARKit. Windows 10 supports eye tracking to navigate parts of the OS, while some game designers use it for more natural navigation in games. This tech has been slowly crawling to the foreground for a while now, but we could potentially see eye tracking as a standard in the next generation of VR headsets.
Qualcomm's most recent VR Developer Kit (VRDK) includes a partnership with Tobii, the biggest name in VR-based eye tracking in the world today. Tobii is supplying the know-how, but Qualcomm is building these reference units for developers to build on while manufacturers work with Qualcomm to include this tech in headsets aimed at release next year. Which part of this chain takes responsibility for your safety? Qualcomm makes the reference design and encourages its partners to use this tech, but isn't the manufacturer of record for the actual consumer product. The manufacturer is probably going to install a largely pre-made version of some other company's operating system and rely on third-party SDKs to enable access to this hardware. Developers will take this information and build lots of exciting things, but it's not immediately clear how this data is collected, handled, or stored.
In sitting down with Qualcomm this week, it was clear no one was really prepared to answer this question. Qualcomm doesn't technically make the consumer product, but it is clearly highlighting eye tracking as a great potential feature. And with good reason, eye tracking can allow developers to collect "heat maps" to show where users are looking and interacting. That makes fine-tuning an experience much easier, making it possible to quickly make an experience much easier or much more complicated. It's not hard to make eye tracking in this context sound deeply creepy, and quickly turn users off to this kind of feature. On the other hand, if privacy settings for this kind of feature is a simple on/off setting, it can quickly deprive users of a much more immersive experience.
With developers unable to access these features yet to see how much data they have access to, and manufacturers not yet announcing commercial products with these features onboard, the only company willing to discuss privacy in VR eye tracking is Tobii. While embedding sensors into the face area of a VR headset is fairly new, Tobii has seen tremendous success with its eye tracking camera for Windows 10. By default, this eye tracking tech does not allow the developer to access any images of the actual eye. Instead, the sensors convert your eye position into a set of coordinates, and the developers can use the coordinates for the appropriate positional data.
If you want access to more than the coordinates, you have to sign a very different agreement with Tobii. As Tobii's Business Unit President Oscar Werner explained to us:
User privacy is very important to us. All of our standard license agreements with developers say that they cannot store or transfer eye tracking data, which is required for analytical use. If a developer wants to store or transfer eye tracking data they have to sign a special license with us. These special license agreements demand that the application clearly inform users that information is being used for analytical use, explain how it is being used, and require that users consent to such use.
Users will have the ability to agree to the use of data above and beyond simple coordinates being shared with developers before any information is offered, which is great. What this doesn't address is eye authentication, something Qualcomm is boasting as a potential feature for partners to take advantage of. For that, SOMEONE IMPORTANT made it clear the eye image captured by Tobii goes into processing to generate the signal and the coordinate, and then is immediately destroyed. "The eye images are never transferred to developers or stored in persistent memory on the device. Developers only get the signal with gaze coordinates (where you look)."
It's going to be a little while before we see eye tracking in VR headsets, and that's exactly why we need to have this conversation right now. A slip up which exposes retinal scans from a batch of users isn't something that should be allowed to happen in the first place. Every company involved in the process should be asked what specifically it is doing to keep this user data safe. And until we like the answer, we should keep asking.
