One of the biggest criticisms of Apple's platform management over the last few years is that, more often than not, they've released as many new bugs as they have features. Now, that's not entirely true. If you look back over the last decade and a half of forum posts, you'll quickly realize every year is the worst year ever. That's because old pain fades while new pain… it sears. But, 2019 has been bad. iOS 13, in particular, came in capital H O T hot, from a sketchy prime seed through a rollercoaster beta and into release.
So much so, iOS 13.1 went into beta before iOS 13.0 came out, and since then we've gone through iOS 13.1.1, iOS 13.1.2, and iOS 13.1.3 at a breakneck pace. And, frankly, more are needed.
https://twitter.com/reneritchie/status/1185635913726119937
Apple is typically aggressive when it comes to the number of new features they add and not aggressive enough about landing them all. iOS 12 was different, though. Apple deliberately pushed back some features that had been planned for iOS 12 and, instead, re-tasked some of their best and brightest engineers — engineers who had helped create some of the modern foundations of iOS — to go back and optimize and improve those foundations. The result was… terrific. Not only did performance improve, especially on older devices, but iOS 12 itself was rock solid from beta through release.
I super high key hoped Apple would make that the new normal and this year would be very much like the last. Instead, Apple went back to the old normal and maybe even tried to make up for lost time. The result was… the opposite of terrific.
Now, iOS 14 is already ramping up. Marketing is pushing down new features they feel iOS needs to be competitive and compelling next year and, engineering is pushing up features they think would be really cool and just as compelling to make.
That's why, most years, by now, I'd be giving you my own wish list full of must-have features, new and carried over, that I really want to see in iOS 14.
This year, though, I'm only going to push out one big wish, one biggest of ticket items alone. At least upfront: Change the way iOS is being developed.
Why iOS 13 is buggy
Earlier this week, former Apple engineer David Shayer, writing for TidBITS, enumerated why iOS 13 and macOS Catalina are, as he put it, so buggy.
First on the list is overloaded feature sets leading to schedule chicken.
Basically, Apple takes on too many new features each year. Too many to finish, much less polish, by launch day. Then, because no manager wants to admit their team's deliverables aren't on schedule, not enough features are deferred in a timely manner. And that causes a lot of last-minute misses.
We have had a few years, like iOS 12 and, of course, OS X Snow Leopard, where the reduction of new features in favor of better performance was headlined as a new feature. But, that they were headlined shows how few and decades between they've been.
It's one of the rare cases where Apple's 1000 nos just aren't enough. They need like 2000. Enough to provide push back against overloaded feature sets and cover for managers who need more time.
Second is that crash reports don't identify non-crashing bugs.
In other words, you can have a low or no number of bugs that cause crashes, but still a high number of bugs that cause frustration. If you're not somehow tracking those as well, things can look better than ever on your dashboard even as you're pissing off your user base on a daily basis.
And humans often respond more viscerally, more viciously even, to annoyance than anything else.
This actually came up a few of years ago on John Gruber's The Talk Show Live at WWDC 2015 with Phil Schiller.
With every release there's bugs, and there's things we hit on, and there's things that the team's passionate about getting out there and fixing.But we're also very careful about tracking crash logs, and AppleCare calls, and Genius Bar visit, and we even have a tool that is able to follow a lot of user forums to ascertain what the complaints are, and try to really gather a good metric, set of metrics on all the issues.And in this case, I do think the storyline isn't really accurate with the reality. Not to say there aren't bugs, there aren't things driving some people crazy—there are. Of course there are. But it isn't a change.
Third is that less important bugs are triaged.
Apple has a classification system for bugs. P1 is major. P2 and P3, increasingly not so much. When engineers are first building a new feature, they can just fix bugs as they come up. When they go into the early stages of beta, there's still time to fix most things major. When they're about to be released, there's only time left for the showstoppers.
That's less a problem than a reality of any large-scale development process, even those at the biggest and richest tech companies in the world. Resources are simply always more limited than the always growing demands placed upon them.
And, since the next year brings the next set of features, the only time engineers can go back and fix older, lower priority bugs are when they're expressly given time in the schedule to do just that.
Like with iOS 12 and anything that affected performance.
Fourth builds on that — regressions get fixed but old bugs get ignored.
What this means is, new bugs that break things get fixed. Old bugs that don't break things get left to haunt the code until they do.
Like, for example, ancient audio and casting bugs coming back to terrorize new audio casting products.
It's not universal across teams, and it's certainly practical in some cases, but bugs like bills have a way of always coming due.
Fifth is automated testing gets used sparingly
WebKit and Safari are famous for zero regression. Any code checked in gets tested for performance and, if it slows things down in any way, it gets checked back out.
Here's Don Melton, former Director of Internet Technologies at Apple, explaining it on the Debug podcast:
Guy: One of the things you keep hearing about the Safari project is that you have performance-based tests. If a commit makes something slower, then it gets yanked.Don: Yeah.Guy: Was that your doing?Don: Yes.Guy: I can imagine, when a deadline is looming, you may be tempted to let that slide a bit.Don: I never did. There were times when I was the most hated person on my team for that. This is actually the point of my talk next month, it's that that is the key. You can never go backwards. That's the Safari secret.
I'm not sure where Apple is or isn't doing enough automated or unit testing, but Josh Shaffer, who spearheads a big part of the future of Apple development, SwiftUI, recently spoke about its importance on John Sundell's Swift podcast.
Testing is just such an important component of building a great app or framework or whatever you're writing and great unit testing and performance testing has been a core element of the development philosophy of SwiftUI from the very beginning.Every commit that we make to the project includes unit tests covering you know whatever new or fixed functionality we have with that change and we run all of the test during code review for every change as it's being made.
That's a good sign. No amount of internal QA can ever equal millions of customers hitting the software in millions of different ways, but testing does get rid of the low hanging targets before they hit them.
Sixth and last is ballooning complexity.
Back in the day, Apple only made Mac software. Then they added iPod. Then iPhone and Apple TV. iPad and Apple Watch. Now we even have AudioOS on the HomePod and BridgeOS on the TouchBar.
What's more, even now, some poor bastards at Apple not only have to still compile iTunes for Windows, but TV app for Samsung's Tizen, and, eventually, all the different Smart products it'll run on.
That's exponentially more to build for, to test for, and to solve for day in, day out, year after year.
And, as a good friend of mine likes to point out — complexity isn't the same as technical debt. Technical debt you can pay down. Complexity tends to accrue.
So, how can this all be fixed? Can all of it even be fixed?
The (Potential) iOS 14 Solution
I fully realize how ridiculous any recommendation my dumb blogger, podcaster, and YouTuber ass can make. But, I'm going to make two anyway. And, hey, if I'm going to run at a wall, I'm damn well going to leave a cartoon shaped hole through it when I do.
First, the iOS 12 approach should go from being the exception to being the rule.
Software engineering organizations don't scale linearly. Especially not when the scale is massive. The overhead always scales with them. So, even if you're adding engineers, as you increase platforms you have to decrease new and updated features per platform to account for that overhead. But, you also have to also increase maintenance and optimization for old features as well, or the new ones risk toppling the whole thing.
That's what made iOS 12 so great. It still had new features, just a more constrained — dare I say more traditionally Apple-like — number of them. But, it also allowed for the time needed to improve performance and reliability. Paying down technical debt, sure, but also deliberately reducing complexity, redundancy, and moving upper level hacks down into better planned, system-level components.
Jonathan Deutsch, former Engineering Manager, on the Debug Podcast:
I think [OS X Snow Leopard] 10.5 had a legitimate number of issues, and I think it was a good call to do 10.6 in that manner, but very specifically, I said 10.6.8, 10.6 had massive issues when it shipped, and when you think about the fact that 10.6.8 was a great update, you had to get through 10.6.1, 2, 3, 4, all the way to 8, and that was a long period of time. Apple wasn't on the yearly release schedule.I think 10.6.8 probably went out with two years of refinement over 10.6, which was, I think, another two years of refinement over the 10.5 update. The 10.6.8 had been begging to get to that point for almost four years,
Second, Apple should move from a yearly update to a yearly roadmap.
Let me explain: The WWDC keynote and September events are just too big for Apple to give up. And I don't think they should. They're great for developers and even better for customers.I just think Apple should change that one slide at the end from "coming this fall" to "starting this fall".
Instead of Craig Federighi listing off the 8 to 12 tentpoles that will all hit customers all at the same time, he lays out the same tentpoles that will all hit customers over the course of the next year, starting in September and finishing in June, right before the next WWDC.
It already kinda works this way anyway, it's just the result of running downhill and desperately trying not to trip and fall, instead of picking a slope and a more measured pace to get to the same place.
We already get the big .1 emoji update in the late fall. You know, the one that really drives updates. We even already get previews of features coming later, like Portrait Mode back in the day and Deep Fusion this year.
And we already get staged released, but for features that just aren't ready in time, like iMessage Sync or iCloud Folder Sharing.
So, just plan all the features out that way to begin with. Take advantage of the beta to make sure what's finished for September is rock-solid in September, and the rest get baked until October, March, even June.
Sure, some features will still have to be finished in time for the new products that depend on them. But for the others, set expectations that they might take some time… and then take that time.
But, those two things — Make every year a half a Snow Leopard year, and instead of setting expectations for a release date, set them for a road map, and I legit think Apple will see a lot less frustration and a lot more satisfaction from everyone, engineers and customers alike.
