>>The user tests were conducted in a specially constructed room featuring a one-way mirror, so observers could watch the tests without being intrusive. The tests were conducted by a moderator who made sure the user felt comfortable and showed her the basics of using a mouse. Then, with no further instruction, users were asked to perform specific tasks, without help from the moderator, like editing some text and saving it. The moderator encouraged each user to mumble under her breath while doing the tasks, revealing her current thinking as much as possible. Each session was audio or videotaped for later analysis.
Amazing to see the importance they gave to UX/UI and the level of sophistication they had forty years back.
That's still the right way to do UI testing. You bring in people, you give them a list of tasks to do, and you take video of screen and user. Someone then has to watch the video and mark the places where people got stuck, had to back up, or gave up. Then create a highlights reel of every problem seen more than once. The devs have to watch this.
This is generally correct, but it’s often better to go to the user rather than bringing them in. People can have quite different reactions in their own homes, especially if they have specific setup needs, than in a research lab.
True, it's much easier now, and you can get a much wider sample of end-users and a more accurate representation of their behavior in their natural context. It's absolutely worth doing most of your usability testing remotely. But to be honest there's still something powerful about having the development team watch a real person through a one-way mirror that I haven't managed to replicate using video. It reminds me of the difference between live theater and watching a movie at home.
The closest I've come to replicating the impact is to have a conference room where the team sits together and watches the usability test being streamed live.
That description should also supply a warning about a difficulty that UX designers, and Apple in particular, have struggled with over the years: they're going to a great deal of effort to optimise their designs for the first hour of use.
Over-optimizing for the walk-up-and-use experience at the expense of a more realistic subset of usage is sometimes referred to as the "Pepsi Challenge" or "sip test" observational bias. You might even consider it a special case of the "streetlight effect" where people choose their research methods primarily based on how easy it is to collect that kind of data. (When I used to teach UX I had a whole lecture dedicated to alternate approaches to avoid falling into the trap of over-optimizing for novices, things like GOMS/KLM, log analysis, heuristic evaluations, diary studies, cognitive walkthough, surveys, etc.)
In 01981 if users didn't like your design in the first hour trying it out in the computer store they wouldn't buy your computer. Now if they don't like your design in the first five minutes of downloading it from the app store they'll uninstall your app. There's a real tradeoff between ease of use and ease of learning, but software merchants are in a real bind here.
Honestly, a leading zero in human-readable text is a non-solution.
Year numbers are variable length decimal strings, not a four or five digit fixed length field. Any historian concerned with 6 AD-999 AD will likely not include a leading zero in any year figures they use. I don't see why the people of the present or the future should do any different.
Machine readable implementations, sure. Have as many digits as you wish. But for human written material, it just looks weird.
Most of us have probably violated all sorts of good UI principles out of either haste or ignorance, but our crimes are mitigated by the fact that for the overwhelming majority of software the number of users, and hence the number cursing the idiot who designed this crappy interface, is extremely low and/or constrained by the bespoke nature of the application; hence the investment in time and money to perform this level of rigour is not justified.
The only time I've seen UI work carried out with this level of care was for an air traffic control system, where of course the consequences of a poor design can have significant and fatal consequences.
Don't forget flight simulators, the flip side of the same crucial situation, have equally elaborate design UI/UX simulations. And I believe power plants, especially nuclear power, have this type of elaborate design simulation, as well as everything in the (outer) space industry.
Equally remarkable is the amount of stuff we take for granted today that the Mac team got right the very first time.
Had it not been for that user feedback, Windows might have copied the Mac and gone with “Do It” as well and we would be living in a slightly more awkward UI timeline.
I always think about this when I look at current Apple's UI. Just a little user feedback would have gone a really long way.
That's still possible today, but nobody can actually be bothered, because there's plenty of other stuff to do as well.
I do recall ~8-10 years ago, we were building an app and did a user test; we had a phone with a webcam rigged to it so we could see from the user's point of view.
But for this you need dedicated (in multiple senses of the word) UX people, and they are both rare, and companies seem to be reluctant to hire them.
Tesler was a computer scientist. He worked on AI and Smalltalk at Xerox, and was a manager at Apple at this time.
What we need to do instead, is bring down the wall between design and development that was built up in the past decade, encourage developers in the UI field to have multi-disciplinary skills, and stop assuming every minute not spent coding or in meetings is wasted.
As a UI engineer, I wish it was more common for UI engineers to learn about accessibility and how to contribute to doing UX research, rather than trying to pick up some back-end and calling that "full-stack". There's much more useful cross-pollination to be gained when branching out that way.
That would be the equivalent of a car designer complaint that engineering ruined it. You cannot produce quality work without that collaboration in either direction.
Form over function? Or beautiful but useless (snark intended) The citrus press by Philippe Starck comes to mind. Many bars own it and display it but don't use it because it sucks. The last sentence came from a bartender who has it on the shelf.
They just hadn't figured out that one can get results of the same quality from a usability testing room without one-way mirror. I don't blame them, the standard literature on how to run tests on the cheap appeared one decade later.
The goto place is the Nielsen Norman Group¹. You can get any of their books, it will describe how to do tests. It's basically catch up to 5 users from your target audience, ask them to do the tasks you want, observe without saying a word, and don't ever come out with any variation of "users are stupid" explanation because even if they are, they are the ones you are creating the software for; fix your software, and repeat.
But if you want details about why those large "record the user" sessions aren't worth it, you'll have to dig into their papers.
Amazing to see the importance they gave to UX/UI and the level of sophistication they had forty years back.