Doug Russell talks to Marc, Seth, Dave, and Rene about the importance of adding accessibility features like Voice Over and Assistive Touch to iOS apps, how it needs to be included alongside the design and development process, and where to go to get help.
Here's the audio, again, in case you missed it. And now, for the first time, here's the full transcript!
Iterate 41 transcript: Doug Russell and accessibility
Rene Ritchie: Welcome to Iterate. Tonight we are doing a special discussion on accessibility in iOS. Joining us, as always, is Marc Edwards. How are you, Marc?
Marc Edwards: Pretty good. Pretty good. I don't know much about this topic, so I'm going to be listening very keenly.
Rene: Seth Clifford. Welcome back, Seth.
Seth Clifford: Thank you. Thank you.
Rene: We have the host of Unprofessional, Dave Wiskus. How are you, Dave?
Dave Wiskus: I am so great. [laughter]
Rene: And we have developer-at-large, the freelance Doug Russell. How are you, Doug?
Doug Russell: Hi, how's it going? Good.
Rene: Once again, we have yet another podcast instigated by Dave Wiskus, which is something I'm incredibly fond of. Dave, what gave you the idea for this as a topic?
Dave: Mostly Doug came to me bitching about the stuff we didn't talk about on the last episode.
Doug: I like to think that I politely brought up the topic as something you could expand upon.
Dave: That doesn't make for a very good story. [laughter]
Marc: Bitching is fine.
Doug: I kicked his door down, and I was like, "We're going to talk about blind people. This is happening."
Dave: Look, you son of a bitch.
Rene: You rage-casted.
Dave: Instead of saying, "That's interesting feedback, and I'm going to ignore it." I said, "You know what? I'm going to give you an opportunity to fix this problem. Let's go back on the show. You come with me, and we'll talk about this out in public."
Rene: You made it his problem instead of it being our problem. I love that.
Dave: Right. It's one of those teach a man to fish kinds of things.
Doug: Teach a man to go on someone else's podcast.
Dave: Right. It's like that.
Rene: One of the things that Apple never gets enough credit for is accessibility. There's all this talk about areas where Apple does well or does poorly, but accessibility is something that they have routinely, year after year, invested heavily in, Doug.
Doug: Absolutely.
Rene: What is it about accessibility that you've been working with or what is it that makes it a good feature for design consideration?
Doug: At a ground level, accessibility is a broad, general term for assistive technologies, which is things that help people who are impaired in some way to use software. If you have a physical or cognitive or hearing or visual impairment, these are the tools that help you use software. There have been screen readers for years. Windows has JAWS and OS 10 has VoiceOver and these kinds of things. But the-in iOS 3 they added VoiceOver to iOS. It is a screen reader in the traditional sense in that it speaks to you. But the interface is completely different from anything that had ever really come before, in the way that a lot of iOS things are very different.
It's a touch model where you turn on VoiceOver and all of a sudden instead of regular touches it's an on-screen cursor, and then you're flicking around or moving your finger around to select the thing, and it's reading useful information to you. Then you're double-tapping instead of single-tapping and select things and there's all this extra information and hints and cuing. There's special modes for readable content, all these kinds of stuff.
It took it from this very niche thing that was hard to use and frustrating and incredibly expensive to something that comes for free on a device that millions of people have that's incredibly easy to use and just pretty much out of the box. Like, if you're a regular iOS developer using regular controls, your app is probably more or less totally accessible already without you doing anything. That was a really major step forward.
Rene: I remember because I used to cover mobile round about the time Apple got into it. But I was previously a Treo user and I'd played around with BlackBerries. They had accessibility about as far as you could get frustrated with them and throw them at a wall. This was a big move, bringing accessibility from giant computers to small, mobile devices.
Dave: I just turned on an iPhone and you're probably going to hear it in the background.
Doug: Just so you know, two-finger tap to make it shut up. [iPhone reads aloud information]
Rene: It's Siri. Siri is now telling you how to use your phone.
Doug: Right. It is exactly the same voice. It's guiding you and it's telling you where you are and where you're going. It makes discoverability really easy and it makes knowing the action that you're about to take... This is one of the basic principles of universal design, which is sort of the term for trying to come at accessibility as a way to make things more available. One of the ideas is tolerance for error. Like if you're on a normal interface like the way that Android does accessibility is you kind of just have to hope you figured out the right thing. Then you tap on it and the tap, the interaction is fairly normal.
But with iOS, when you jump into it, you have a huge amount of ability to-oh, I'm on the wrong thing, I'll go to the next thing. Oh, I overshot it, I can go back. That kind of stuff, like the tolerance for error is very high which is really a pretty important thing if you don't have vision.
Rene: I don't know if you want to break it down or whether it's worth breaking down, but just looking at the accessibility menu under settings, general, there's VoiceOver, zoom, large text, invert color, speak selection, speak auto-text, hearing aids, LED flash for alerts, mono audio, guided access, assistive touch, home-click speed, incoming calls, and triple-click home. Wow.
Doug: Before I talk to that a little bit, there's the idea of physical motor disabilities and visual disabilities and hearing disabilities and stuff like that. That's why you get so many different things. Like zoom is built-in. That's simply that you can three-finger double-tap. I don't use that one a lot, but zooming is literally zooming the whole screen in, and then you can pan around. That's just going to make everything big. There's no developer requirement there.
Dave: Zooming is not just useful for disabilities.
Doug: You can pixel peep.
Dave: That's useful for me.
Doug: [laughs] Designers like it a whole lot.
Rene: I was about to say I didn't want to give you any trade secrets, but every time I've seen Dave hit a new app, the first thing I've seen him do is zoom in and start examining pixels.
Doug: Yes. You can actually pan up and down to make it zoom in and out by how much it zooms, so you can really get in there and be like, "Hey, your drop shadow's a little tiny bit wrong. Fix it."
Dave: I say that a lot.
Doug: All the time. Speak selection is actually something that's interesting because that's one that's really useful for just about anybody. If you turn speak selection on, any time you select text up in that little UI menu popover thing that shows up, one of the options will become speak. You can select a whole article, hit speak, sit your iPad down, and it'll read it to you. It's like built-in text-to-speech that's entirely distinct from the normal VoiceOver system. It's a useful thing to have.
Speak auto-text is a really poorly named feature because that, when it will actually do a replacement, it will say it out loud. If you turn that on by accident, you're bound to be really annoyed. You can turn it on for your friends' iPhones if you want to confuse the crap out of them.
Dave: [laughs]
Doug: Actually, on that one, when you turn on VoiceOver, the whole touch model changes. When you go into the VoiceOver screen, there is a...VoiceOver has to be on. Once VoiceOver's on, there's a VoiceOver practice menu option, which is pretty useful. You can go through and learn all the gestures. One thing you should know about is that three-finger triple-tap will black out your screen, and the three-finger double-tap will turn off the voice. That sounds like a lot of stuff, but if you walk up to someone's iPad who has VoiceOver on and you three-finger triple-tap, it'll turn the screen off. Then double-tap, and it'll turn the voice off. Now their iPad appears to be completely broken.
[laughter]
Rene: Do not do that at home, kids.
Dave: This is the best WWDC session I've ever been to. [laughter]
Doug: Right, so don't do that, but that's sort of a thing I've seen happen to a few people. Then there's mono audio. Mono audio is another thing where there's really not a lot of developer intervention there. For example, you're writing a game and there's stereo audio, and it doesn't collapse down to mono well -- for some reason, the way you've mixed your audio, when it collapses down to mono, it sounds really bad, or it cancels out in a really ugly way -- you might need to swap your audio out when someone turns on mono audio mode.
Marc: You definitely shouldn't be mixing your audio in a way that doesn't work when collapsed anyway.
Rene: You shouldn't do that anyway.
Doug: You shouldn't, but people do.
Marc: That's really bad.
Dave: Unless it's a game that does things with audio intentionally as part of the gameplay mechanic.
Doug: If you're doing binaurals, like you put the headphones on and it appears that it's in 3D, but you mixed it wrong, and it collapses down to mono incorrectly there's cancellation and stuff like that. Some games will do this. Although games in general are fairly poor for accessibility, which is unfortunate. I probably will mostly focus on VoiceOver, but there is Guided Access. Guided Access is incredibly useful for parents and very useful for...You'll be in classes with kids with autism and things like that. You're using educational software, and you need to make sure they can't get out of the app.
Guided Access, when you turn it on, you can go into an app. You activate guided access mode, you enter a passcode, and you can actually draw over which parts of the screen they're allowed to touch and which parts they're not. It'll turn off the home screen and turn off the power button or the sleep button at the top and all that kind of stuff. You basically have a device that's kid-proof.
Rene: It's funny because as much as there is in accessibility, and I'd like to actually break down a lot of that with you as the show goes on, Apple has made this such a focus the last couple WWDCs when they introduce marquee features like FaceTime, it had someone doing sign language over FaceTime. For Siri, they had someone who was visually impaired having messages read to them and replying over Siri. Those were emotionally resonant, but they were also an example of how even mainstream features in the phones also had accessibility aspects.
Doug: Siri is a huge breakthrough. If you are a low vision or blind user and you need to send a text message, the interface before was all right, but typing is not cool. When you have VoiceOver mode on, you tap a letter. It tells you the letter. You double tap. Then it types that one letter. It takes a really long time to do anything useful. With Siri, you hit the button. It says, "Hey, dictation." You're in. You speak a whole paragraph and then bam, it's in. Then you say, "Send," and off it goes. That's a massive breakthrough if you're a person who's visually impaired or actually completely blind.
Rene: Basically, was that a big hint from Apple that this is something that you should be paying attention to whether you're developing, designing apps or whether you're approaching a phone for someone who has those requirements?
Doug: I couldn't speak to their internal politics, but I certainly hope so. It's something that they clearly really care about. My interactions with the people on the accessibility teams have all been really positive. I'm not sure what Apple's motives are, but it's great that they are featuring this stuff.
Dave: Did you see thing about...What was it? Samsung going after Apple for patents on accessibility features?
Rene: Yes.
Doug: I'm actually not aware of that.
Rene: Yeah, they're suing them in Germany. Right, Dave?
Doug: Oh, jeez, really?
Dave: Yeah. [laughter]
Dave: The thing was that it's like, "We have the patents on how to make things work for blind people." I don't know. Part of me feels, maybe, they do have the patents, and maybe, they do have a legal claim to this and, maybe, they are technically right, but man, what a stupid move. What a terrible PR decision.
Doug: Because Samsung cares about PR?
Dave: Oh. [laughs]
Marc: Yeah, maybe, they don't, but certainly...
Doug: ...That's crazy, though.
Marc: ...in terms of benefiting society that seems like a bit of an ass move. [laughs]
Doug: You're not making any friends doing stuff like that.
Marc: No.
Rene: No. Conversely you had that Tim Cook moment where they were showing someone using the iPhone to navigate a forest that they would never otherwise be able to navigate. Both those things have a very different...
Doug: Right, yeah. No, if you've got your CEO up on stage reacting to that video the way that any well-minded human being would which is he clearly got choked up at the end. If you got one company suing the other going like, "You can't help blind people." On the other side you got Tim Cook having this intensely human moment. That's a pretty bad clash.
Rene: What point of the design process is accessibility a factor that you start to consider? Is it from the beginning? Is it something that you start with the UI work?
Marc: Yeah, look, it definitely seems like something you should consider right from the very start. I have to preface this by saying our apps are really quite bad at this. [laughs] I, certainly, can't speak as an expert. We've certainly made sure our website is accessible, but apps is something that we haven't addressed well enough at all. Except I guess, we have some color blindness testing in Skala view which is pretty handy. We do test for that, but we don't test for total blindness. Which is a shame, we really should. That's why we're here talking about it.
Doug: Right. [laughs] It's definitely worth taking into mind. It's something that a lot of...Designers are naturally going to be visually oriented people so the idea of thinking about what is and isn't good audible design is a really complicated topic. It's something that's poorly covered. HTML accessibility, like you were talking about saying that you're website's accessible. The idea of what is and is not an accessible website is fairly well documented. There are the basics like Apple provides an accessibility programming guide, but all that is very dev-centric. It's very like, "Here's how to go into objective C and write a line of code that does a particular thing."
Not like, "Here is what a useful amount of information is. Here's what's too much. Here's what's not enough. If it's this kind of information, we would expect you to annotate it this way." For that stuff, it's mostly left you to wing it. It's not a well-traveled thing.
You guys talked a little bit last week about automotive design and things like that. Like the Ford SYNC and stuff like that. I think there's, probably, some value in thinking about what they're doing there because they have some of the same problems. You can't take your eyes off the road so you have to interact through this audio system. I feel there's, probably, parallels to draw there as far as looking for design information.
Marc: Yeah, I'd say many parallels. It's, essentially, the same thing. It's just the user that's really changing.
Doug: Right. There are a lot more people who drive than who are blind. One of the biggest problems, here, the total blind population is not very big whereas tons of people drive cars.
Marc: Yeah. The other thing we discussed last...
Dave: Really, nobody's going to make a joke about blind drivers? [laughter]
Marc: Go for it Dave.
Dave: Thanks.
Marc: The other thing we did touch on last time was the MYO, that armband thing that you can control via gestures, but it picks up the impulses in the movements before you even move. Obviously, that's another technology where there's people with disabilities using similar devices today that were, probably, ridiculously expensive in the past and, like you said, not very well supported. Whereas this is something that everyone can use and it's something that benefits mainstream users, but, possibly, has a lot of other offshoots for people with disabilities. I guess, this is a very similar thing.
It's something that previously was for a very niche specialty group that is now mainstream and giving everyone benefit. Which hopefully, helps everyone a little bit, but helps some people a lot.
Dave: Isn't the goal really to eventually not have to use your screen for anything that isn't a visual interaction anyway? If you're watching video, I guess, sure, or if you don't want to interrupt the people around you, sure, but shouldn't the goal be to interact with our technology through electrical impulses in our skin and through verbal communication anyway?
Doug: I don't know. Imagine a classroom full of kids talking to their iPads while they're trying to...
Dave: Well, I'm saying if don't want to disturb the people around you then, of course, the same reason you'd watch a book rather than...read a book rather than watching a movie?
Doug: Sure. Mark was talking a little bit about the idea of doing things that are helpful to everybody. There are some basic tenets. This is the crux of what they call universal design which is, "Figure out stuff that is really useful for everybody, and then make sure that it helps these other people." There's stuff like you can never make your text too big. If you have resizable text, there is no size...if you get it where there's one word on a page, that's awesome for somebody. If you can design things that resize, or if you can figure out defaults that are slightly different. If you have a set of things you can control...
Like if you go into Instapaper which is an app that has very good accessibility, it's got a full screen mode where the top and bottom bar disappear. That's hostile to voice-over because the buttons disappear, and it doesn't tell you the buttons are gone. Now, they're no longer discoverable so it has a different setting that you can go and hit in the settings and say, "Leave those there so that it's useful." That's just sort of a thing.
It's not a thing in the settings that says, "Hey, voice-over users, this is for you." Some people don't want that bar to go away. It happens to, also, be really useful for people who use voice-over, so stuff like that.
Seth: Dave, going back to what you were saying about the future of interfaces and using your voice, and our touch and stuff like that. I think there's always going to be a delineation between the tools that work most naturally for us. I think that if it's certain types of things where our speech would be the natural output of our thought process, like conversing with people or delivering a concise set of instructions to a machine that can perform an action, I think definitely, I'm right there with you, that that's the most natural way to go. Imagine trying to perform spreadsheet functions with your voice.
Dave: Oh, of course.
Seth: There's always going to be stuff, whether there are people around or not, where it's just not feasible to do it unless we have brain computer interfaces hooked up, and at that case, we're working twice as fast as we are now. So that's a whole different story.
Dave: It makes me think of...people always talk about the iPad feeling like something from Star Trek. There's always going to be connections to things that we saw in sci-fi as a kid. The thing to me in Star Trek that stood out, relevant to this conversation, is when Geordi would be in the Holodeck trying to solve some engineering product, and he'd be like, "Computer, what if I did this?" and he'd start running a simulation just by spouting stuff off as he thought of it.
Seth: Yeah.
Doug: Yup.
Dave: To me, that's way cooler than just being able to do things on a Holodeck, like being able to think through at that level.
Seth: Oh yeah. At that point, you're basically problem solving with another entity. You're basically standing with a person and bouncing ideas off them and they're giving you results that you can then move into other ideas and other patterns, and that's a very natural thing for people to do. That's definitely something cool.
Doug: Why I'm waiting for Jarvis.
Dave: Yeah, instead of going into Photoshop and, "Well, I'm going to tweak this, I'm going to tweak this." I just want to be like, "Hmm...what would it look like blue? OK. Now a little bit more blue, less blue, less blue."
Rene: More beautiful blue.
Dave: Let's try red. Nah, that doesn't work.
Seth: Make this pretty.
Rene: Seth, I was just curious. You do a lot of contract work. NickelFish does a lot of contract work. How often does accessibility come up in conversations, and is it something that clients themselves raise often?
Seth: I've been doing a lot of thinking about this lately, and I'm actually glad you asked. It's not. It really isn't, and it should be because the types of people that we work for are servicing a much wider audience than we think of when we're building apps. It's definitely in their best interest to consider that stuff. I've spoken to our developers a while ago. I think probably when we first started thinking about this several months ago and it came up, I had a conversation with Justin about it and I said, "I want to talk to the team and get them thinking about this and try to have this be a thing that's in the forefront of everybody's mind, so as we're building, not only are we trying to do things that make sense, but we're doing things that will benefit this other population of people that's large that really could use these same tools and in an effective manner."
I've mentioned it. I brought it up. I probably need to continue to talk about it, but more to your point, clients really have not brought this stuff up with us and it's one of those things where I don't know if it's because timelines are already tight and they think it's going to be an extra effort or if it's just not on their radar or if they're not thinking about it, but I'm sure that they must have some volume no matter how small or large of call center requests or something from people in this position where they're like, "I can't use your app. I really want to use your app."
So I've been thinking about it. I want to start bringing it up and just feel them out and see what the situation is, what their landscape looks like for this because it is important. I've been thinking about. In many cases, as we were saying, it's not hard to do if you're building things the right way from the start.
And in terms of building good interfaces for users, period, one of the things we try to do is make apps that make sense. I think that the first step in making things accessible is not over-complicating the application in and of itself, thus it lends itself more easily to any user whether they're sighted, not sighted, hearing impairment, whatever.
Doug: Yeah. Information architecture makes a huge difference in how accessible something is, because if it's well laid out and really easy to get through, then it's enormously easier to make it accessible.
Rene: That might be a good test of information architecture.
Doug: No, it definitely is. I would say a few things about what you were saying before, which is that a lot of clients see it as a line item that they can drop, like, "Awesome, there's a thing in there, I can chop it off, that's great." There's things like the Americans with Disability Act. That's section 508 I think. That's where government agencies and things like that, if they're contracting work, they have a legal obligation to make their software accessible and Apple and these kinds of things.
But if you're random entertainment company five, you don't have a legal obligation. There's no...the way that Target got sued years ago for their website not being accessible, nobody's gotten sued yet for their app not being accessible. So a lot of places just go, "Yeah, I don't got to do it."
But if you get it in at the start, like if you're testing regularly and thinking about whether or not your software is going to work for people who have these kinds of disabilities, it's generally pretty easy to make it work. It's just sort of some basic stuff. Make sure you're using real text instead of text and images and make sure things are labeled and all that kind of stuff, and then you're not really adding a lot of extra work. You're just sort of making what you're doing continue to work.
Marc: Is there any resources you've seen online, just in terms of putting a dollar value or a percentage of users on this stuff? It seems like it would be an easier sell to a client if you can say, "Hey, well, there's three percent of users here," or whatever.
Doug: I don't really have numbers handy, but the one thing that is really useful is there's a website called AppleVis, which is an incredibly active community of low vision and blind users who actively review and talk in detail and comment on software whether it is or isn't accessible, they recommend it to each other and this is an incredibly active group of people. It's really great. So if you write something and you make sure they know about it, they will let you know quick whether or not you're doing something wrong and what you could be doing better. It's hard to beat these kinds of people for engagement.
Rene: What about apps that use a lot of...a lot of big companies seem to just throw their app in a UI web view and call it an app. Do they gain anything?
Doug: So, fun fact. If you take a web view and then you throw it in a scroll view, that is something that absolutely works, but is not allowed and crashes a lot of the time under VoiceOver. A lot of those apps just blow up. If they work, they're all right. The web views accessibility is generally pretty good. People don't tend to do stuff. So there's this thing called ARIA which is the W3 standard for how to go through and annotate web text to be like, oh, this is a list of whatever and it's got items organized in this way. You can really annotate your HTML really nicely.
Most people don't bother with that, so you'll get stuff where it's like they've made an image for an anchor instead of actually giving that text and that kind of stuff. It's almost the exactly the same kind of things you see people doing with UI buttons in native apps.
Dave: So you're saying if I tie these two threads together, you're saying that if I threaten to sue my bank over their shitty app not being accessible, they might fix it?
Doug: If you actually sued them, they might fix it.
Dave: Oh. I don't know if that's worth it.
Rene: That was exactly the point I was going to bring up is that a lot of these companies...you see this a lot with passbook apps as well or bank apps or movie apps is that they've built these secure web facilities to handle transactions and they are either too lazy or don't know how to recode them as native apps so that you just get that show. And those are the apps that you need access to the most often because it's your important data.
Dave: Absolutely.
Doug: So the flip side of that, though, is not necessarily banks, but because the built-in controls are so good, if that company decides to just go ahead and build a native app, even if it's fairly simple, a lot of times, the crappy mobile Bank of America site or whatever -- and I actually don't know what the accessibility is like on Bank of America -- but theoretical bank accessibility could be really bad on the web and their incredibly simple iOS app might actually be much, much more accessible.
Marc: Yeah. I'd even settle for bank input text fields that can handle additional spaces and dashes rather script that stuff, so I guess the chance of them getting this stuff right for vision impaired is...
Dave: My favorite is getting the -- what do you call it? The automatic capitalization from autocorrect in my username.
Doug: Oh yeah.
Marc: Yup.
Rene: The original version, I forget if it was the Blackberry Storm. I think it was the Blackberry Storm started autocorrecting my passwords. Marc : Oh cool. That's really handy.
Seth: That's always impressive.
Rene: That's the thing. A lot of these apps, you have to enter passwords and stuff like that. How is that handled from accessibility?
Doug: That one's a little dodgy.
Dave: You just say it out loud.
Doug: It actually says the letters out loud. It's pretty easy to turn it off. You'll go in and you'll triple click your home button and turn off VoiceOver and type in your password and triple click. I guess I didn't really cover this earlier. A huge number of people that you would just sort of refer to in a group as blind are actually just low vision. If they can pick up their iPad and if it's got decently large text, they can read it. So for those people, they'll just kind of hold it close and tap away. If you're dealing with it where you're completely blind or low vision enough than you can't see it, then you'd better get some headphones because it's going to say your password out loud.
Rene: So what kind of things do you start thinking about when you're putting together an app? Is it risky to just assume iOS will do it all? And how much of the process do you try to keep an eye on or actually manage step by step?
Doug: So mostly that's about how much stock UI you're using. In the same way that you might do flows for wireframes, it's not a bad idea to sit down and do a flow of what information you have available. For example, textual information, make sure you have actual real text, and if they're sending you texts HTML, make sure that you have a way to represent that that strips out all the tags so it's not reading the tags and stuff like that. But it's mostly just sitting down and saying, "What information do I have to represent this?" and because a lot of the apps that I build tend to be server driven, am I getting enough context information from the server? If they're sending me images, do I get a description of that image that I can attach to the image use so the user will know what kind of image that is?
And then figuring out how to write really good summary text. Effectively what you're doing is applying labels to everything. So sitting down and saying, "What is a good description of this?" I keep going back to Instapaper, but if it's Instapaper, the grids say the title of each article but it doesn't try to read the little bit of body text that it shows. It doesn't declare how much of the article you've read because that's sort of extraneous information you can get once you dive inside of the article. Stuff like that.
Marc: I'm guessing all the label text, that that would also need to be localized as well.
Doug: That's a really good idea. It's not mandated, but it's a great idea. Absolutely.
Marc: I'm guessing vision impaired for non-English speaking would probably be even more of a disadvantage. Certainly for US-based software anyway.
Doug: Definitely. No, that's absolutely true. Hey, I've got a button, right? So I'm going to call this the read later button. You could just call that read later button and it will actually say out the words "read later" and then it'll pause a little bit it and it will say "button" but it's way more useful, and this is what Instapaper does, if it says, "This is the read later section or the liked section or the archived section." So you don't have to have the labels on the buttons that VoiceOver reads match the title of the button. The sort of automatic behavior is to take either the title you give it, or actually the name of the PNG which can be pretty funny. If you go fishing around in the app store, you can find apps that read really random names for buttons because they're using the name of the image asset.
But it's better if you go in and you say, "OK, I'm going to annotate this thing." If I'm building a tweet view list that's got random tweets, I might say tweet users name X, tweet body content. Body content's no good, but tweet content, whatever. Just so that they know exactly where they are in the thing.
So I'm working on a podcast app and you go through and it's got a number on it for what episode number it is, but if all I did was leave that label to just say the number, all they know is that there are random numbers on this screen. If I go in and I replace the accessibility label to be show number 1700 then that's way, way more useful. That kind of stuff. Extra metadata to give them context.
Rene: You want enough so that there's an idea of what it is but not so much that it's extraneous and it confuses them?
Doug: Right. You want to make it so they can skip. The primary navigation mechanism is flicking, so you can flick left and right just with one finger and that takes you between items horizontally, across left to right and go to the next row and so on, and that's the primary navigation mechanism. If you give them a ton of information, now they can't skim effectively. You have to find enough information but absolutely no more than necessary.
Rene: Skim-able is the audio equivalence of glance-able, I guess.
Doug: Exactly.
Rene: You mentioned earlier that games specifically do a bad job. Is it that they do a bad job or is it that the way they do it is more difficult, like an openGL viewer?
Doug: No, it's because they're written in openGL so they don't get anything for free. It is possible, it's just very hard.
Dave: How would that work? Most games, do they even lend themselves to that kind of control. Could you tell Angry Birds?
Doug: My accessibility whipping boy is Letterpress, which would be really, really great if you made it VoiceOver compatible, but he wrote it all on openGL. I don't know. Loren Brichter is a super genius, but he also does stuff he doesn't need to do like rewrite UI Kit in openGL which means that he doesn't get any accessibility for free which means he has to do all of this work on top of it.
Seth: He has to rewrite open accessibility in openAL to make up for it.
Doug: Right. He might write his own whole thing. It's the kind of thing where he did this really amazing technical feat and the cost of that amazing technical feat is that the app is completely useless for people who don't have vision. Games like Letterpress or Scrabble or any kind of card game are great for people with low vision and when people build these kinds of games and don't make them accessible it's really unfortunate because you're basically just saying to a whole group of people, "I'll get to you eventually maybe." That's no fun.
Rene: Is accessibility a sell, then? It sounds almost like if someone is not doing it you need to evangelism them, you need to convert them to it. How big an effort has that been, in your experience?
Doug: It's a mixed one. Developers tend to be very keen on it once you actually explain to them how it works like hey, by the way, you can do this and it's not going to cost you the rest of your life. You're not going to spend 80 percent of your time doing this. It's not a ton of effort and you're going to make this group of people unbelievably happy. That's usually a pretty easy sell. Clients tend to look at you and go you're trying to sell me some crap that I don't want.
Rene: That's going to change over time. Clients used to look at design as a line item.
Doug: Yes. Some of them still do. I'm hoping that it'll gain traction. This is the kind of thing that this is a huge amount of selling one-to-one where I get somebody, I'm at a conference, and I'm like, "Do you know about this? OK, let me tell you about it and I'll show you why it's not so bad." I put a bunch of sample code on GitHub and anybody who engages me on Twitter I will help them as much as I can and stuff like that. It's a lot of one-to-one, "Hey you, come over here and I'll tell you about why something's great." There's not a great way to get out and get it in everybody's faces except for the things that Apple are doing, which they're better than nothing but it's not as effective as I would like.
Dave: You don't scale very well, that's the problem.
Doug: No. No. I'm working on it.
Rene: I'll ask this to Dave. Is that something that a developer should sell from the beginning when they're taking a client request or is that something the designer should follow-up on or yes?
Dave: Just like I think design is everybody's responsibility, I think accessibility should be everybody's responsibility. Either you care about the products you're making and you want your clients to understand how much you care and you want them to be involved in the caring process or you don't.
Rene: You gave a wonderful talk at MacWorld and if people haven't seen Dave talk in person yet I definitely recommend it as something that you add to your list of things to do.
Dave: Bucket list.
Rene: Bucket list, yeah. Your design bucket list. You broke a lot of it down to giving a shit, whether you care about the stuff or not, and it sounds like this is another element that you have to add to make sure you care enough about what you're making.
Dave: Right. This is why Doug and I get along so well. We come at it from different angles, but we both care so intensely about the stuff that we make that even when we do disagree, when we did disagree when we were working on stuff together before, it's just so natural to end up in a place where even if we don't think the other's idea is the correct way to do it we both know that we want the same thing which is to make the best possible thing for other people to use. When you're working with people like that, if you come from that place you're naturally just going to make better stuff. It's so easy for people to talk about SEO style tips and tricks and the problem with that is that it's disingenuous. You're doing things because you think you're going to make money at it. I don't know that I would try to sell accessibility to a client based on here's the numbers and here's how much more money you'll make so much as I'd just say, "Look, do you care about this thing? Do you want to make something that's really great or do you want to cut corners just to save a couple of bucks?"
Doug: This is the whole thing, the concrete implementation of good is going to change the giving a shit and then actually executing on that is never going to be a thing you don't want to do.
Rene: Is it the classic Steve Jobs painting the back of the fence point or has this become table stakes now? Is this something that's a nice to do if you're willing to go the distance or is this at the point now where you have to do it?
Doug: I don't see it as optional, personally. I understand that there are things that are never going to be accessible. Angry Birds is never going to be accessible because that's not helpful. Readability is a really good example. Readability has insanely bad VoiceOver support to the point where I'm pretty sure they've just never tried testing with VoiceOver. It is a reading app. This is the thing. There's no excuse for that kind of behavior. That's not polishing on the table. That's saying that a sizable group of people should be treated as well as everyone else. That's just basic treating minorities with some basic level of civility.
Rene: What did you think, and I know this is a little bit off-topic, but when author guilds were angry when things like the Kindle would read their books?
Doug: That was so messed up.
Rene: Because there were no audio book rights assigned to those publications.
Doug: It's still messed up. They still don't have decent VoiceOver support for it. It's really gross. We were talking about Samsung before. It's saying me getting paid is more important than people being able to read books.
Dave: I hate to play this card, but can you imagine if it were we're not OK with women being able to use this app because we want to charge extra for that?
Rene: Gender specific licensing.
Doug: Yeah. It's a little bit different than that but yeah, it's kind of in that. It's a secondary effect. They were concerned about their audio book rights and then this secondary effect is this entire accessibility community is now hosed.
Dave: Which, just like the Samsung thing, even though they might technically be right it's such a painfully stupid decision.
Doug: Oh, yeah. Yes.
Seth: The root of that was a fundamental misunderstanding about the difference between audio books and VoiceOver support for people with disabilities. Rene, I'm pretty sure we had this exact conversation at one point where we were saying audio books is a performance. You've got people reading the book, maybe it's the author, maybe it's actors. That's a different piece of art than the computer actually parsing the text and then spitting it back out to you so you're able to understand what you can't see. They're two completely different things.
Doug: The way that I saw that particular thing go down is that Amazon, unlike Apple, stepped up and said here's a great feature for everybody, not here's a great feature for users with low vision that happens to benefit everybody. I think if they had positioned it like that the authors would have been terrified to launch that suit. As soon as you step up and say feature for everybody they go, "No, no, no, we get paid for everybody. That's not OK". That was just a mistake on Amazon's to ever open their mouth and go, "Look at this thing."
It's a really compelling idea. Text to speech is really useful. Apple could change VoiceOver to make it easier for regular users to use to read books. iBooks could just have a text to speech mode but they don't have one like in the app, it's a part of the OS, and if somebody were to sue Apple for that they would look crazy, but they can sue Amazon when Amazon says, "Here's a thing for everybody."
Dave: I understand the authors' problem. I went to Hong Kong one year and I saw photocopies of one of my books being sold on shelves in bookstores.
Doug: That's pretty cool.
Dave: It is and it isn't.
Doug: Like crappy Xeroxes?
Dave: Yeah. Literally, Xeroxes on the shelves being sold.
Doug: Did you buy one?
Dave: No. I took a photo, though.
Doug: You should've bought one.
Dave: That would've been excellent. I didn't think of it at the time. When this whole thing happened, I'm like yes, listen to it all you want. I want that to be accessible. I even started recording it myself just to make a point. I never finished it, so I probably should. Don't be a dick I think covers whether you're Samsung suing or whether you're the author's guild. Don't be a dick.
Doug: Right. Don't be a dick applies to take a minute and actually test your app with VoiceOver. I'm not asking anybody to give their life to this cause but turn on VoiceOver, see if your app works. If it's totally busted try to make it not as busted and over time try to make it good. There's definitely incremental steps here. You can go from completely useless to fairly useful to really good in a year. You don't have to do it overnight. If you're doing your own product, you just want to spend some time on it, make it better. Any time you get a chance, make it better again.
Rene: Can we delve into assistive touch a little bit because that always fascinated me.
Doug: Assistive touch is amazing.
Rene: Especially with the rotor and everything like that.
Doug: The rotor is different. The rotor is a separate thing.
Rene: Can you explain the difference to me?
Doug: Two things. The rotor is there all the time during VoiceOver. The rotor is actually kind of annoying because they don't make that API available to third party developers. The rotor is basically if you're in a web view you can click through and say I want to go through this thing, when I flick I want it to go to the next line, the next word, the next paragraph. The rotor is how you get into editing mode so if you're in a text editor there's this big, wacky contortion to be able to select some text. I actually can't ever do it. It's really, really hard. The rotor is effectively like a mode. You do a two finger rotation on the screen and it tells the VoiceOver to go into a different mode which things like web views and some of the Apple apps take advantage of. UI text views, when they're in edit mode, will make extra stuff available in the rotor.
Assistive touch is this totally amazing thing they added in iOS 5 for people with physical motor disabilities. There's something you probably wouldn't think about a whole lot but not everybody has three fingers. Not everybody can move their hand far enough to do a two finger swipe from the top of the screen to the bottom of the screen.
What they added is this little guy that you turn on and it shows up in the corner of your screen as a little dot and then you can tap on it and it opens up and has access to Siri. You can fix the rotation of the device like you can say no matter what the device is physically orientated I want it to be pegged to this particular orientation that is not its actual orientation.
It's got a way to get to home buttons. It's got a way to program in custom gestures. If you regularly use an app that requires you to do a three finger circular swipe or something like that and that's just not something you're physically capable of you can program that in and in the bottom left corner just move one finger in a very small circle and it'll move it in a larger gesture. It'll map out to a larger gesture and now you're doing it.
It's effectively a tool for people with physical motor disabilities to do just about anything on an iPad. It's a very totally amazing innovation.
Rene: Is there anything you have to do on the development side to take advantage of that or is that also just free?
Doug: Nope. It is just free, and it's amazing.
Rene: It's interesting because they have the customized gestures and you also have customized vibrations for people who want different senses when different people email them or call them.
Doug: Right. That's for deaf people.
Rene: It sounds like there's a lot. Is there anything you'd still like to see Apple do with accessibility?
Doug: I have a million weird API things. I want access to the rotor. I really want access to the rotor because writing your own custom editor is basically impossible. Writing your own custom editor that is fully accessible in the way you add text to UIs is basically impossible because you can't get to the rotor.
Rene: What would you do with a rotor if you could?
Doug: I would have edit mode. If you're drawing everything with core text so you want to make it accessible so you feed it all the lines and stuff like that but you want it to support extra modes like being editable and stuff like that you can't get to the rotor so you can't click into editable mode so you can't do selection. That particular one is the one that I want the most. In general, the API has been incredibly good since it was introduced, and they're unbelievably responsive. There's a mailing list called Accessibility Dev and email accessibility@apple.com and radars, if you can tolerate radars. They're incredibly responsive. I literally just went and said I could use this thing, it would be really helpful if I had this, and then not that much later it was in the OS.
Rene: If you were to give advice to someone who was starting from scratch, who had an app that was completely inaccessible and they were looking at it and it looked like a Herculean task to them, if you could break it down into the baby steps version how would you recommend they get started with accessibility?
Doug: I guess I'll say that we're probably speaking to designers mostly. If you're a designer and you're going to sit down and you fire it up and you're navigating through it and it's doing nothing then you're in some trouble. If everything selects the way you think but it's not saying the right stuff then that's just setting labels. If it's that you have a big block of text and it can't read it then you can probably go to your dev and find out how they're doing it and it might be that they're just string draw. To circle back to Loren Brichter again, who I am a huge fan of but I'm going to poke at again, he actually wrote a whole big thing when Tweety came out about how to draw text to make scrolling and table views fast. It's true it makes scrolling and table views really fast but it also makes accessibility stop working. What you have to do is then just take one extra step where you set that text on something that the accessibility can find.
It's junk like that where basically what you'd do is sit down and say it's totally broken, I've got to go talk to these guys and figure out how this thing breaks down into pieces and see if we can start applying labels to it or using more standard controls instead of custom controls, especially now that UI appearance is a thing. If it navigates more or less correctly then you would sit down and say this button here is called cancel, let's make sure it actually says cancel when you tap on it.
There's a big block of text. Can it read? Does it read in lines the way you'd expect it to? Stuff like that. Then there's the above and beyond which is sitting down and doing flows, the same way you would do a control flow through wire frames and stuff like that. You can sit down and say if I'm a VoiceOver user how many taps does it take me to get from X to Y? Can I make that shorter, better, easier to navigate for VoiceOver users?
It's possible to detect whether or not VoiceOver is on and to change out things like you could make a navigation thing shorter or something like that or you can make text more explanatory. Junk like that.
Marc: Just to double back on the scrolling performance thing, I'm assuming that was because it was caching a whole bunch of stuff to images which, again, is exactly the same thing if you're designing something and you're making it images anyway.
Doug: It is absolutely identical. It is the nerd equivalent of using stylized text in Photoshop and then baking it into an image and then sending it off to a programmer. On our end we would sit down and say this scrolling isn't fast enough so what we'll do is draw every single thing in a table view cell into one flat bitmap and then send that to the GPU. That's super fast because it's all flat and there's no compositing and compositing is super slow. Awesome. There's no reason not to do that because you can actually, then, say I have this text and I'm drawing it so I'm not using a UI label. All I need to do is tell the cell. I keep talking about labels. Labels is an actual property on objects and you can set a label on anything. Instead of putting a label in a cell and then giving it text, you're drawing inside of the cell so all you've got to do is tell the cell this cell's label is that text. Then when they select the cell it'll read the text and everybody's happy. Your scrolling is really fast, low vision can still figure out what's going on.
Rene: This may sound a little counter-intuitive but when you're doing sound design is there stuff you need to factor in? A lot of times there's vibration instead of tones in iOS but if you're designing a game or you're designing an app is there a certain level of bass or frequency of bass or vibration motor instead of sound that you can think towards?
Doug: I'm not really aware of that. I don't know. One thing you probably should keep in mind is that you wouldn't want your audio stomping on announcements. There are little notifications that will tell you like when something's reading it'll tell you when that finishes reading so you're not blooping over something trying to read a label. I'm not really familiar with that. I don't know. I couldn't speak to it well.
Rene: It just popped into my head so I thought I would ask because you see people dancing sometimes based solely on the bass that they feel in the floor. iOS, of course, builds in all those different vibe tones and everything, too I'm wondering if Tweety is queuing you on a new tweet, not Tweety, sorry, if Tweetbot is queuing you on a new tweet based on a bird chirping or something like that, whether there's custom vibrations that you could design in instead to describe different behaviors without having to have it speak the sound?
Doug: I'm not aware. Tweetbot has only very recently gotten even remotely decent accessibility. They don't really go out of their way to do anything fancy. I don't know. I'm not really sure what their motivation is for adding the audio in the first place. It's kind of fun but I'm not sure what they were thinking when they were like we need audio. I'm not sure how to then translate that into something else that would be useful for users with hearing impairment.
Rene: Twitter developers, man. They have it out for accessibility.
Doug: It's an issue.
Seth: It's exactly the same thing. The scrolling in Tweetbot is insane. It's the fastest thing in the history of scrolling. Those guys are good guys. They're not bad dudes. They just wrote something that did one thing incredibly well by getting rid of the standard way of doing it. They said the standard way is fine but we can do it way faster and then they didn't step back and go, "OK, the standard way exists. What did we lose by giving that up?" What they lost is accessibility for a very long time and now their accessibility is alright. It's not great but it's pretty good.
Twitterific has always had really exceptional support for VoiceOver, if you're looking for an accessible Twitter client.
Marc: Definitely in terms of just getting back to the audio mixing, you're talking about collapsing to mono. Anyone who's doing any mixing, especially if you're mixing something that's high quality audio, just music, you should always be checking for mono. Even though it can sound cool in headphones, you should never be flipping the phase of one channel. It seems like a cool trick until you realize how dumb it is.
Doug: Oh my God, my drums are super great. Oh, wait, it's mono, I can't hear the drums anymore. That's great.
Marc: Even if you're listening on the world's best stereo speakers, the further you stand away from them the more it becomes mono anyway and the more the trebles blend and the more you end up with weird things which will then call different balances and different instruments will be different volumes and you won't understand why and it's all because you're doing this dumb phase stuff. You have to be careful with time delays, too. Again, to do some cool stereo trick you can't just offset one channel by 10 milliseconds and expect everyone to like it. You've got to check for mono compatibility again.
Anyone who's doing that for games should really check themselves anyway. Forget about visually impaired or hearing impaired or whatever. You're killing it for even normal people, so don't do it.
Rene: To bring it back to you, Dave, I'm going to quote that corny Aaron Sorkin line about, "You don't fight the fights you can win, but you fight the fights worth fighting." This looks like something that is definitely worth doing. For my part, I'm going to make sure that when we review apps, we start checking them for accessibility and make that a line item in the reviews that we do.
What do you think people can do in general? There's complaints to developers. There's people who are developing and designing. You can make it a point to bring it up. Is there anything else that you can recommend?
Dave: Make noise. That's really all I can think of to do. That's the democracy of what we do. If we're talking about end users getting a hold of developers, then maybe send an email. Maybe leave bad reviews saying that, "This app is great, but I'm blind and I can't use it," or whatever.
Doug: Start with an email. Don't leave a bad review right away. Nobody likes bad reviews.
Dave: One star. Doesn't have this one feature. [laughter]
Dave: What I'm saying is, bring it to their attention. Let them know that...
Doug: A lot of people don't know about it.
Dave: I don't think that most people are leaving accessibility out because they don't care. They're just not thinking about it.
Seth: An awareness issue.
Dave: Right. I don't think there's malice. I don't think that anybody's sitting around going, "Man, screw blind people." [laughter]
Dave: They're just not really thinking it through.
Doug: That's probably a pretty small group of people. There's like five or six evil villains making apps, and then everybody else is doing [indecipherable 52:16] [laughter]
Rene: There's Blofeld Incorporated, and then there's everybody else.
Doug: Right. But that's like five people, so don't worry about it.
Dave: I don't want to make it seem like there's this anti-disability movement of people who are trying to screw over the disabled. If it does matter to you, if you want these features, you've got to let them know or they're just never going to think about it.
Doug: It's helpful to be constructive. Emailing somebody saying, "Here's my story. Here's why this would be helpful," in a reasonable way is incredibly useful. If you send someone an email like, "You're a jerk. I hate you," they will not help you. There's a surprising number of...I've, in general, promoted people going, "Hey, you should send a letter to those people." Then you see stuff on AppleVis and just in general, where they'll send the email to somebody or post publicly, "These guys suck. They're bad people. Blah blah blah."
They're probably not bad people. They're probably decent guys who didn't know about it. Send them an email, and there's a pretty good chance they'll help you out.
Rene: And it goes back to our "don't be a dick" thing, and it's surprising how much better reaction you can get when you do plead your case in an intelligent and engaging way.
Doug: The human side of this is profound, like just the basic human level, like appealing to the fact that they're probably a decent human being, saying, "I want to use your software. I want to be a fan. I want to be a person who gives you money and buys your software and follows what you do, and I want..."
Rene: And recommends it to all my friends.
Doug: Right, yeah. And this is a very, very engaged group of people. Like low-vision users who have found out about iOS and are just like, "Holy crap, this completely changed my life!" Like Matt Gemmell wrote a really amazing post about the fact that it's like a bionic extension of you that completely changes the way that you interact with the world. There's not a lot of opportunities in most people's lives to be like, "I can do a trivial thing, and make a bunch of people's lives immensely better," there's not very many things like that. Accessibility is one of those things.
Dave: Maybe I'm being a little cynical here, but if you want your reason to do this, it's because especially right now, so few people are doing this that if you do it and you get a few emails from people talking about how great it is that they -- who are low-vision or deaf or whatever -- are able to use your software, that's a huge PR win for relatively little effort. If you want to be cynical, even that is a good reason to go down this path.
Rene: Doug, thank you very much. I think this is a hugely important issue. Dave, thank you very much for bringing it to our attention and bringing Doug to our attention, and making this part of the conversation, because that is how we hopefully increase awareness.
Dave: I like to help. [laughter]
Doug: Everybody's always talking about how nice Dave is. [laughter]
Rene: And now he's accessibly nice, too.
Doug: Yeah.
Rene: So Doug, if people want to find out more about accessibility, if they want to get help with their apps, you listed a bunch of sources. We'll put all those in the show notes. If people want to find out more about you and the stuff that you're doing, where can they go?
Doug: I blog at TakingNotes.co, which doesn't get updated a lot, but does have some stuff about accessibility and just programming in general. And then I have a GitHub that's just called GitHub.com/Rustle, spelled R-U-S-T-L-E, /accessibility, and it's just a bunch of stuff that's not super easy to do as example code. So if you're just sitting there going like, "Crap, how do I do that?" there's a pretty good chance I have some sample code there. If you're trying to do something and I don't have sample code there, let me know, there's a pretty good chance I will just build it and put it there for you just because I want people doing this. And then I'm on App.net as Rustle, spelled the same way as before, and then I'm on Twitter as @halfliterate, where you probably shouldn't follow me.
[laughter]
Dave: Hang on, I'm still writing all of that down.
Rene: I'll put it in the show notes, Dave, I'll make it easy for you.
Dave: OK, thank you.
Rene: You can go to imore.com/category/iterate and find all the show notes. There's a five by five spot there. Dave, where can people find out more about you and your independent work?
Dave: Oh...All over the Internet. I'm many places. @dwiskus on Twitter and App.net and whatever else there is. My blog is betterelevation.com. For music, airplanemodemusic.com and unprofesh.com for the podcast.
Rene: It is one of my favorite podcasts. I really appreciate that you do that every single week. It's among...
Dave: It's one of my favorites too.
Rene: You have to say that. You're bought and paid for, sir. [laughter]
Seth: I'm @sethclifford on App.net and Twitter, and if you want to see some of the work we do, nicklefish.com
Rene: Marc Edwards.
Marc: I'm @marc on App.net. That's Marc with a C. @marcedwards on Twitter or bjango.com for apps.
Rene: You can find me @reneritchie on App.net or Twitter. You can find me at iMore.com. For all of our shows, you can go to mobilenations.com/shows.
