From today's criticism to tomorrow's Siri operating system, Apple is best positioned make AI deeply personal, deeply integrated, incredibly consistent, fully mesh networked, and — eventually — into a full-fledged platform for the future.
Sponsor: Thanks to Skillshare sponsoring this installment — the first 500 of you to use the promo link http://skl.sh/vector will get your first 2 months for 99¢.
Transcript
Phil Schiller (V.O.): For decades, technologists have teased us with this dream...
Runner: Read me the message.
Siri: New message from Sebastian.
Schiller (V.O.): ...that you're going to be able to talk to technology and it'll do things for us...
Runner: Reply, "Definitely. I'll see you there."
Schiller (V.O.): ...Siri.
Runner: Play my running mix.
Rene: I love Siri. I use Siri all the time. Probably half my interactions on my iPhone are done now through Siri. It was, and is, a delightful Pixar-style character [laughs] come to life that just helps me get through the day. I use Siri on my Mac to look up things, or how to spell words, so I don't have to stop my workflow or take my hands off the keyboard. HomePod is the closest I've ever come to ambient computing.
Where, no matter what I'm doing -- working, cleaning, cooking -- I can just ask Siri to turn on my lights, or check the temperature, or set an alarm, or play some music. I never have to stop doing what I'm doing. The more I use Siri, and the more I come to depend on Siri, the harder I'm hitting up against its limitations. These limitations are frustrating. Apple had such an early lead in voice-first interface.
I really think if Apple would spend some of its famous focus, if it would prioritize Siri...It is uniquely positioned to make Siri the next generation artificially intelligent assistant we've all been dreaming of -- since we saw "Her," or JARVIS, in "Iron Man" -- to lead in voice interface for the next decade the way it's led in graphical user interface and multi-touch interface for the last couple of decades.
To do that, it's going to have to take a few painful steps.
Siri: Make Siri more personal.
Rene: It's not enough for me to say, "Where's the new Marvel movie playing?" and for Siri to get better and better at learning that that's what those words mean. Siri also has to get better at knowing that I prefer Marvel movies to DC movies. It has to get to know not just what I say, but what I like.
With HomePod, if you say, "Siri, play music I like," it's using all of the music that I've added to playlists, that I've liked, that I've asked it to play, to form a better more personal knowledge of me. This needs to extend to everything. This also means that we need voice ID, not just on iPhone or iPad, but everywhere that we have Siri. It can't know who we are if it doesn't know who it's listening to.
It's something Apple started to do with the iPhone 6S, but it didn't bring it to HomePod. In a personal device context, it's what allows for security and privacy. It knows to let my voice in and keep everyone else's voice out, but in a group environment it's what enables multi personal assistants.
It's what lets it know to read my messages to me but not my roommate's messages, to differentiate between a child's request for Raffi music [laughs] and my request for Public Enemy. Without being able to do that, not only can't it provide the services we need, it can't provide the understanding of us that the service needs. Apple prioritizes security and privacy, and it gets dinged for that.
It makes people think that it's impossible for the company to build deep knowledge about you. Here's industry Analyst, Carolina Milanesi.
Carolina Milanesi I definitely think that there's two sides. One, is that there's a lot of stuff that I'd be prepared to share with Apple, there's a lot of stuff that Apple already knows about me. It seems that people just think because they are so focused on security and privacy they cannot know about you. Not true. There's a difference between knowing and sharing what they know, which is the other side.
Then, there's how they can learn. The level of data that is private, and then the level of data that is public, and how you take those two and use them together to give me the service that I want.
Rene: Right now, Apple's doing very little of that. They are synching your Siri dictionary between devices. They're using a secure version of CloudKit -- which is the engine that powers iCloud. They're not taking all the data about all the choices and all the decisions that you make on your iPhone or iPad or Mac all day every day and using that to better inform Siri.
That's what Apple has to do in order to make Siri a next generation artificial intelligence.
Siri: Make Siri more worldly.
Rene: Then there's the rest of the world. That's what Apple's been addressing with SiriKit, but SiriKit took far too long to come out, and it's been far too slow in progressing. We have a few domains -- we have to-do apps, and note apps, and ride-sharing -- but there are whole ranges of services that Siri still doesn't connect with. Apple says they're building out these domains and these intents slowly...
So that we have a robust way of communicating with Siri. Right now, there's very little to show for it. It doesn't feel like enough resources, enough focus, is being devoted to SiriKit. That's a shame because Apple has a huge opportunity here. Amazon was first to market with Alexa Skills, but they've almost painted themselves into a corner. Here's how Brian Roemmele, Voice First advocate, explains it.
Brian Roemmele: The idea of using skills and keywords is a dead end. Let's look at the domain system of the web. After all the great domain names were taken up, people kind of got depressed. Then we said, "Well there's a .net, .org." Then they started inventing all these other domains, but now there's a confusion. Who owns the right domain?
Is it an .IO domain? Or, is it an .AI domain? Or, is it .com? There's only one .weather domain on Alexa. There's only one .flower domain, or .Uber domain...that's a brand. Let's say, ".taxi." Let's say, ".pizza." Who owns the .pizza domain? Well, the first person that wrote the pizza app. Is that the best app? No, but they were there first. Should that dictate what should own the domain .pizza?
Well, no. OK then, we're going to take it away from the developer who worked their ass off to get that pizza app -- which was maybe the best that they can do -- and what, sell it and give it to Pizza Hut or Domino's? Is that fair?
Rene: [laughs]
Brian: We know it's a dead end, so how do you deal with that? The only way you can deal with it is you've got to walk down that one-way road backwards and say, "Oops, we shouldn't have gone down here, it's a dead end. We have to redo the whole thing." Apple's got the advantage today to do it the right way.
Rene: Given the opportunity, it feels like SiriKit should be a crown jewel in Apple's developer frameworks. Hopefully, the company's starting to come around to that.
Siri: Make Siri more consistent.
Rene: Turn the lights green.
Siri: Done.
Rene: Turn the lights purple.
[ding ding]
Siri: Purple coming right up.
Rene: Turn my lights blue.
[ding ding]
Siri: Blue it is.
Rene: Turn the lights natural.
Siri: Sorry. I can't help you with HomeKit here.
Rene: No human being should be expected to remember which Siri device can handle which query. Consistency is an end-user feature. If you can't rely on something, you stop using it. They have to all handle all queries and never not give you an answer. Now, context is important. If you're holding an iPhone or an iPad in front of you, Siri can use the screen.
It can be less verbose and more visual. If you're using a HomePod, or using CarPlay while you're driving, either you don't have a screen or Siri can't rely on the screen. It'll have to be less visual and more verbose. Lia Napolitano, who worked on Siri User Experience, explains it like this.
Lia Napolitano: One of my first projects at Apple was going through and saying, "What are all of the things that Siri says?" Then, "If you're in a car, do all those things work?"
For the longest time, if you got Safari results from Siri it would say, "Here it is." That was it.
[laughter]
Lia: For a number of reasons, it's completely unacceptable if you can't look at your phone. We had to go through each of these experiences and say like, "What is the likelihood that you're looking at a screen? What is the likelihood that you're looking at it, but you can't touch your screen?"
You realize that Voice is not perfect. Voice isn't great. We cannot read all of the web results in a perfect summarized way.
[crosstalk]
Lia: ...the way you like it. We don't know how much time you have on your hands. That's where visuals are a great compliment. I think Siri is at its best when it is beautiful and resonant and has personality, but also helps you get your stuff done more quickly than rifling through a lot of UI.
Rene: Either way, Siri has to always answer. If it can't answer -- given the constraints of a device -- it has to push you to another device.
Siri: Make Siri mesh networking.
Rene: Here's industry analyst, Ben Bajarin.
Ben Bajarin Even the iPod, you could argue, was probably a precursor to this, the value of just having music at your disposal wherever you are. Now these things manifest themselves in new ways because they're all over your home. The benefit here is this theme that we talk about called, ambient computing. Some device that's listening to you, some device that you can have a conversation with.
Some device that might even know more context about you, what room you're in...If it's got a camera in it, it might know a little bit more about you, what you're doing at the time, and feeding value back to you from a more contextual environment in this kind of ambient area. That could manifest itself in a speaker, in a wearable, in something that goes in our ears, in our cars.
Rene: Google, Rene Ritchie.
Siri: I can help you search the web for Rene Ritchie on your iPhone. I found this on the web for Rene Ritchie, take a look.
Rene: Now, mesh Siri is the idea that the main intelligence lives, not outside your devices, but across your devices. Outside your devices, rings a lot of alarm bells about servers and who's owning and who's controlling and who's accessing, your data. That's what makes mesh such an appealing solution. Siri shouldn't have to push HomeKit from the Mac to the iPhone. The Mac should work [laughs] with HomeKit.
It shouldn't have to push Calendar from HomePod to iPad. Calendar should work on HomePod. In instances where you're moving between devices -- for example, if you ask for directions on HomePod -- you're not going to take your HomePod in the car with you but you are going to take your iPhone. Any time you ask for directions HomePod should offer to push them to your iPhone.
The same way Apple Watch offers to push Google results to your iPhone. If the lack of a screen prevents HomePod from doing some advanced email function or calendaring function -- and it knows you have an iPad close by -- it should offer to push those tasks to your iPad.
The same way mesh routers around your house always make sure you have the best Internet connection depending on where you are, mesh Siri would make sure you always have the best functionality given whatever devices are available to it.
Siri: Just make Siri OS.
Brian: I would get teams across Apple together, internally, and say, "We now have Siri OS. It's its own platform. It's going to live and die on its own. It's going to touch everything that we do. I'm going to pipe all of the teams inside of Apple together on an AI [inaudible 12:32] system, if you will, that AI's going to mediate everything that we do from now into the future."
What Siri OS is about, it is an AI-mediated OS. It connects all of these different ontologies and taxonomies that we're building. Mac OS is going to tap into it, iOS is going to tap into it, but primarily, our voice is going to mediate it.
The next level would be we need to open this up to a developer community to a level that no other system's ever been opened up [inaudible 13:07] voice-based system. We need to be able to allow developers to, in real time, build what Workflow promises.
Rene: For a long time we had punch cards from mainframes locked up in IBM Vaults. Then we had command-line interfaces, things like the Apple II or the DOSBox. Then we moved to graphical user interfaces with Windows and Mac PCs. Now we have multi-touch interfaces in our iOS devices and Android devices. With each leap forward in personal computing, we get a leap forward in how we interact with those computers.
While screens aren't going away any time soon, it's inarguable in an increasing number of contexts, Siri has to become the primary way that we interface with our devices -- whether it's ambient computing at home or at work, whether it's controlling things as we drive or pilot in a not yet fully autonomous world, whether it's through the AirPods in our ears that lets us keep our phones in our pocket.
Or, whether it just allows us to have multiple levels of interactivity going on while our hands are otherwise busily interacting with our iPads or Macs. Siri OS is what's going to take us into the future. It's hard to imagine Apple launching the Mac as a platform with only a couple kinds of apps supported. It's hard to imagine the App Store launching for iPhone with only support for reminders or note-taking apps.
All of these things had to be robust platforms, so that we could enjoy robust functionality. Siri is the same thing. It's only when everyone -- from developers to users -- can do almost everything with Voice that they can do with touch or typing, that we'll start to understand the potential of this technology, that we'll start to achieve the next generation of computing.
I don't want to downplay any of this. It would take an incredible amount of work for Apple to make Siri into a proper platform. It took an incredible amount of work for Apple to make the Mac into a platform, to make iOS into a platform, and they did it because they knew the future of the company depended on it.
Not everyone's going to agree on this. Not everyone inside Apple is going to agree with this, [laughs] they fight and argue the same way that everyone outside Apple fights and argue...
Rene: Different points of view win at different times. It's becoming increasingly clear -- to me, at least -- that the arguments supporting Siri have to win, and I think they will because not getting super serious, and not shipping Siri OS, will be like not shipping the Mac or not shipping the iPhone.
It won't just be about not being competitive, it'll be about not being relevant in the next generation of super accessible, super personal computers. If there's anything that has dominated Apple's history, it's been their relentless drive to make evermore personal, evermore accessible computers.
Schiller (V.O.): Siri can speak with your natural language. It's conversational. It understands context of the things you're talking about as you're following along. It's personal in that it's amazing right out of the box, but it actually even gets better as you use it and it learns your voice.
Dancer: Hey Siri, play me something I'd like.
[music]
