At CES we checked out the Structure Sensor, which aimed to provide 3D reconstructions of the real world on the iPad well before Google's Project Tango was announced. With this iPad attachment, you can quickly generate models of faces and objects, and also interact with those models in augmented reality apps. That means after a quick scan, you could play a game where you have to walk around real-world objects to reach virtual objectives with your iPad. Other, more practical use cases included scanning whole rooms to generate floor plans, or sending models off to 3D printers for reproduction. We got a chance recently to play with the Structure Sensor ahead of the September retail launch.
Android users are all excited about the upcoming Project Tango, a tablet device which Google is working on that will also dynamically generate virtual 3D models using a depth-aware camera system. The Structure Sensor operates in a very similar way: a laser emitter peppers the view in front of it with infrared dots, while a second camera picks up where those dots are hitting, and generates a view of how far they are relative to the device holder.
Let's dig in and figure out if the Structure Sensor can offer iPad owners the same thing Tango is doing on Android.
Hardware
The Structure Sensor itself is very nicely built and available in blue or steel finishes. Four screws attach it to the device-specific mount, which comes separately with a little screwdriver. The structure needs to be charged up separately and holds about 3 to 4 hours of active charging on its own without tapping into the iPad battery. The sensor is also plugged into the lightning connector on your device to send the data to whatever app you've got running. That cable gets in the way a little bit, but not overly. For developers that want to get really down and dirty, an optional Hacker Cable is available that unlocks new functions in the sensor, and allows a live stream of imagery to be shunted over to your Mac or PC.
The bracket has a simple clip mechanism that makes it extremely easy to attach and remove the Structure from your device. Brackets are available for the iPad Air, iPad mini, and iPad 4, plus the standardized screws being used allow others to make their own brackets for other devices with minimal fuss. A gap is left for the iPad camera to continue operating, which is something of a necessity for most of the apps. The bracket covers the sleep button, but still offers full access with pass-through button.
Software
Completing scans can be finicky, depending on the software you're using. For best results, you need to have even lighting and enough space between you and your subject. If tracking of the object is lost, it can be a hassle to get it back and often requires you to start over. It doesn't do well with reflective objects. Often you'll get RAM errors, suggesting you close some apps so a Structure app can run. An irregular background is helpful and sometimes necessary for the sensor to orient the 3D model as you circle around it. The fidelity of the final .OBJ files might not be on par with something hand-crafted by a proper artist, but it provides a strong starting point and can be easily tidied up with the right tools. Besides that, the general shape and look of an object may be all you need, depending on the work you're doing.
There are four open-source sample apps currently available, plus a third-party app and the core Structure app which provides a raw view from the camera. The two augmented reality apps, Ball Physics and Fetch, work well enough, and demonstrate that object collision and occlusion (that is, the view of the virtual character being blocked by real-world objects) works just dandy. There's certainly novelty and promise there, but nothing hugely compelling as is.
The Scanner app, though simple, will likely provide the most utility to users. You set a target area cube by pinching the screen, then start a scan. Once you've made a few passes from multiple angles, you can e-mail yourself the final .OBJ file, which can be opened in most big-boy 3D model renders. It would be great to do some editing of the model directly on the iPad if an app supports importing. So far the only one I've found that advertises the ability to do this is Forger, but it crashes every time I attempt the import. Unfortunately Adobe's 123D Sculpt only exports for now.
The only third-party app currently supporting Structure is itSeez3D. The UI is polished, allows exporting as .PLY or .OBJ, as well as sharing out to the popular free 3D model network, Sketchfab. Unlike the sample Scanner app, itSeez3D automatically takes individual pictures, which can be a little jarring when you aren't in position to take a steady shot. Though Sketchfab is great, my scanned Xbox 360 controller didn't look so hot. I tried taking a scan of an Olloclip lens attachment, but the rendering failed, presumably because it's too small. Larger objects, like this sleeping puppy, turned out much better. Even my creepy 3D selfie looks pretty good.
Bottom Line
Scanning stuff with the Structure Sensor is definitely a unique experience. It's like you're quietly spackling everything around you with a whipped cream hose and walking away with the hardened shell. Needless to say, that's pretty fun. The initial models seem a little fuzzy at first, but when the texture is applied, they seem much closer to reality. When you pare the models down to their wireframes and you can see every individual triangle, you start to get a real appreciation for how much data is being gathered in so little time. This technology has been in use for awhile, but packaging it in an affordable and an accessible way is a noble goal, and these are fine first steps in that direction.
The biggest problem with the Structure Sensor right now is that the use cases are somewhat limited. Occipital is eager to recruit third-party developers to create compelling apps that make use of this technology, but at $399 for the hardware (nevermind the cost of the iPad itself), it's hard to imagine anything but industry-specific niche applications being made. Augmented reality gaming has yet to take off, and this expensive accessory isn't likely to entice end consumers. Even within professions with conceivable use cases, the Structure Sensor faces steep competition for developer interest against Google's Project Tango.
For those interested, you can preorder the Structure Sensor for iPad here for $399, or go for $499 and get access to the Skanect desktop SDK.
