A future powered by Augmented Reality has more and more captured the imagination of the tech world in recent years. In truth, Augmented Reality is very much here, manifest in more niche fields like sports and "extra" features like Apple's LiDAR scanner for iPad and iPhone 12, Apple's own first serious moves into the AR sector. Apple itself is rumored to be making a play when it comes to both AR and VR products, and rumors of an 'Apple Glass' product, as well as various rumblings of a VR/gaming headset that have started ramping up in recent months.
There are many challenges when it comes to AR; size, form factor, technological constraints, not to mention databases, and privacy. Many of these are yet to be fully conquered, which is why, for now, AR remains beyond the grasp of mainstream users. One man who knows a thing or two about AR is Dan Eisenhardt. Founder and CEO of AR company FORM, Eisenhardt was born in Aalborg, Northern Denmark. A swimmer since the age of six, he spent his early 20s getting degrees in Mechanical Engineering and Business before a trip to Vancouver started an adventure that would put him on the path to AR innovation.
Tasked with pitching an idea in one of his entrepreneurship classes, Eisenhardt came up with the idea of augmented reality swim goggles. The basic concept? What if a swimmer could see their metrics whilst they were swimming, stripping away straining at the pool clock between lengths or reliance on a coach and numerous stopwatches. Eisenhardt's pitch drew enough attention that he formed a like-minded group of students to run with the idea. But there was a problem.
Eisenhardt was pitching this idea in 2006 when there wasn't even an iPhone, let alone compact AR technology that you could wear on your face whilst swimming. The challenges were numerous, as were the costs.
"There was the cost side," he tells me, "the form factor side, and the complexity side as well. Going into the chlorine water in the pool at that form factor and then finding a way to get an accurate detection from the head because your head is not moving that much. And then machine learning."
Now a mainstay of technology, Eisenhardt says that there weren't many machine learning engineers around back in 2006, making it impossible to create a product that could accurately measure a swimmer's movements from their head whilst in the pool. With both market and technology risks, the project quickly revealed itself to be unfeasible. Yet all was not lost, and, encouraged by their university instructor, the group decided to pivot to another application as if they really were a real-world business.
Based in Canada, and with the 2010 Vancouver winter Olympics four years away, Eisenhardt and his team created Recon Instruments, a company that would go on to release five generations of heads-up display goggles worn by skiers the world over. Four of the original team of six helped start the company and worked for seven years to pump out digital heads up display technology that was included in big-name brands like Oakley and Uvex. Their products were even sold in Apple stores worldwide. Just a couple of months after the launch of its Recon Jet cycling sunglasses, Recon Instruments was bought by Intel in 2015, and Eisenhardt became the general manager of the heads up display division at Intel. 18 months later, Eisenhardt had left Intel and returned to the idea that had started the journey in 2006, AR swim goggles.
Eisenhardt and FORM worked for four years to create its first pair of AR swim goggles, overcoming many of the previous challenges that had been insurmountable more than a decade earlier. Just a few weeks ago, FORM's AR Swim Goggles arrived on Apple.com and in the company's retail stores. Not just a gimmick, Apple's backing of this product proves that FORM might well be the next big thing in swimming, just like it did for his ski goggles. When Recon products started showing up in Apple stores, Eisenhardt says it became the company's most successful channel and that people began to realize that its AR skiing goggles weren't just a gimmick or a gadget but a fully-fledged product to be taken seriously. Eisenhardt was able to call on contacts from Recon's stint in Apple stores when it came to FORM, too, building on the fitness emphasis Apple has been pushing with products like Apple Watch and its recently released Fitness+. "It's much better to be in the Apple store as an accessory if you're not more expensive than the main product," Eisenhardt jokes, recalling how Recon's ski goggles often retailed at around $600; by comparison, FORM's Swim Goggles are only $199. But at $199, who are these goggles for?
Target market
"Triathletes really love this product," Eisenhardt says, "this is right in their wheelhouse. They're already expecting real-time metrics for running and cycling." Whilst Apple Watch compatibility and integration is a huge boost for FORM, the company wanted to ensure integration with the most popular watches amongst fitness enthusiasts like triathletes. Garmin, he says, is the most popular. Where Apple Watch comes in is in bridging the gap between FORM and fitness swimmers. "We're not the elitist, you know, the brand that only works for the absolute best swimmers," Eisenhardt explains how FORM is a brand that should appeal to everyone, that meant leveraging integration with Apple Watch to reach millions of fitness swimmers who use devices like the Watch to elevate their swimming workouts to even greater levels. FORM doesn't need a companion device in the pool; it uses machine learning and the head tracker to measure your swimming, outdoors it can track both GPS and Heart Rate when paired with an Apple Watch, the ultimate one-two tech punch for your swimming workout.
So how do you make a set of AR swim goggles that appeal to everyone but are good enough for elite swimmers? The answer is testing, lots and lots of testing. And great materials too…
"We wanted to appeal to any kind of swimmer like we really want to make swimming a source of joy for everyone," says Eisenhardt, "but in terms of getting the accuracy up high enough to appeal to competitive elite swimmers, even Olympians, that was something we aspired to from the beginning so that we could say we could satisfy their needs, even though they're not going to be the biggest customer group."
Through hours and hours of testing with one of Canada's best swimming teams, the University of British Columbia Thunderbirds, FORM worked on fit, form factor, and the algorithms it uses to track workouts. Hundreds and hundreds of goggles were 3D printed for testing as the team tried to balance aerodynamics, comfort, battery life, and more, finally reaching the optimum. Beyond UBC, FORM tested its goggles on fitness swimmers of all ages to ensure accurate workout tracking for whoever decides to use its Swim goggles.
The numbers
With no mainstream competition in sight, the swimming market is ripe for the taking. Eisenhardt rattles off the numbers to me; there are 240 million swimmers worldwide, making it one of the world's biggest sports. In the U.S. alone, there are 30 million active pool swimmers, one-third of those swims once a week. The other two-thirds are fitness swimmers like the aforementioned triathletes, the market, it seems, is huge. But it's not easy; like any good video game, Eisenhardt says that swimming is easy to pick up but difficult to master. It's also one of the most aspirational sports out there.
But what about when we step out of the pool? Where does the future of AR and VR lie, how close are we to the everyday, mainstream AR that many dream of or have seen in the movies?
Out of the pool
"Personally think AR is so much more exciting than VR," says Eisenhardt, "because VR is great for gaming but you're isolated, right?" "The whole idea of AI is that it's a mixed reality thing, you are immersed in reality, you have situational awareness," he continues, "you can then choose to get information overlaid, that makes your decision process better or your experience is enhanced."
This drive to bring information closer to our attention has not only fuelled AR development, says Eisenhardt, but plenty of our advancement as humankind through the centuries. "Moving the clock from the church tower to your wrist, that took a few hundred years, right? But then suddenly, you have time that was independent of location." Now communication is mobile, Eisenhardt says. Next, we want to make it instant and ubiquitous, building a completely new set of experiences that will distract us less and enable us more, freeing us from the grind of unlocking our phones a hundred times a day. The prize, Eisenhardt says, is "gigantic," whether for health, entertainment, navigation, or anything else. So how far off are we?
It depends on what you mean by AR. As noted, through Recon and now FORM, Eisenhardt has helped bring AR into the mainstream in unique verticals like swimming and skiing, but the lightweight, thin glasses we can wear without noticing? That, Eisenhardt suggests, is the product most people have in mind when it comes to augmented reality and the breakthrough which will signal that AR has arrived.
"I think it will start in the home," he says, "but that, of course, is a very tall order for multiple reasons," like the form factor and battery life. More focused on enterprise and the military, a product like HoloLens is a great example of the beginnings of this tech, but also reveals how far off it might be: "[It's a] very powerful device, but they have spent a lot of money, and it's still a pretty big device" he cautions. Trying to shrink that to a pair of glasses that weighs 15-20 grams, or even 40, is a huge undertaking.
The other big challenge is data. Also very close to being solved is the huge amount of information required to create databases so that AR can be useful in everyday life to identify your surroundings. He explains it to me, waving a pen in front of his webcam as he talks, that a human could identify a pen from any direction, but for a computer to do that would require massive amounts of data. So you have trillions of objects and countless angles they can be viewed from, couple that with the raw processing power required to make those real-time calculations (probably whilst moving), and the numbers really start to add up. Tesla, he says, and its autonomous driving technology is getting close, though, and it is something that's being solved. The other big challenge, noted by Eisenhardt and plenty of other onlookers, is privacy. In a world less private than ever, but where privacy has never been more of a concern, how do you regulate a significant portion of the population walking around with a camera on their face, filming everything they see and do? Eisenhardt says that even if we successfully solve the technological challenges and the informational challenges at hand, there will still be a big hurdle to face regarding privacy and perception of such a new and potentially invasive technology.
But as Recon, FORM, and Eisenhardt have proven, whilst there are still hurdles, AR has never been closer. Indeed, rumors persist that Apple plans to unleash both an AR and VR headset into the world in the coming years, perhaps as early as next year. Could this be the mainstream AR breakthrough we've all been waiting for? Only time will tell.
