Testing an interface with people utilizing paper prototypes is a great way to encourage feedback and push your design in new directions.
Sometimes testing prompts you to make major overhauls to your interface because you are exposed to a critical flaw or limitation. You draw, remake your paper prototypes and head back out to see if you’ve hit the mark. A major benefit to paper prototypes is that they help you nail down the major paradigms for your interface before you get entrenched in a particular approach.
In Rapid Ideation and Creative Problem Solving we’ve been using this type of iterative process to design interfaces for a thermostat (concept map, version 1, version 2, version 3). The major paradigms for my design were nailed down early: I wanted a device with a limited number of interactions that emphasized the most common uses of thermostat to be intuitive, quick, and easy. I also wanted a design that wasn’t going to present as another screen in the home. That meant avoiding rectangular designs. There’s no doubt that I was heavily influenced by the Nest thermostat and I’ve tried to maintain Nest’s usability and compelling design as a bar for my design in this project.
You can check out some user flows as well.
In the last few rounds of testing my design has been more about refining details and creating cohesion across the device. For instance, in the latest round of testing I realized that one trouble spot for people involved an inconsistency in how the interface was displaying information versus controls. In order to exit sub-menus, users had to use the dial to select EXIT (below).
I had attempted to draw consistency between the controls in this submenu with font size, weight, and curvature. But that relationship wasn’t immediately registering for some users and even though they were able to problem solve their way through it, I want the interface to be as intuitive as possible given the major paradigms I’m working under. I think at least part of the confusion has to do with the way the top of the interface presents a unified circular set of controls and that makes the user feel like the top of the device is where they have control and the bottom is informational. My previous version was actually reinforcing this impression because the interface offers prompts related to pressing the display in the same region where the EXIT appears (see below).
I have seen in previous testing that a few prompts like the one above helped users gain authority and confidence in the interface very quickly. Rather than scrapping my approach to the bottom of the display, I decided to include a new hint to the user to indicate areas they can use the dial to select (see below).
It’s extremely satisfying when you can learn from testing incidents, identify an issue or limitation, and design a solution that unifies and strengthens your concept rather than forcing you to compromise some aspect of the interface.
Deciding When to Move Forward
There is no perfect interface that will be intuitive for every user. Consequently, adjusting an interface through testing can be a never ending process. How good is good enough?
One way of answering that question is to take that which is qualitative (the overall experience of using an interface) and attempt to quantify it. In our testing each week, our users are filling our forms that rate their experience of the interface through a series of questions they can agree or disagree with.
Their responses are then scaled to an overall scale of 0-100 and this is broadly referred to as a System Usability Scale (SUS) score. In my latest round of testing my interface scored an average SUS of 92 (compared with an average of 82 from a previous round). But the SUS scores feel more like a backward justification than a rich indicator of progress.
Like most things in my brief experience as a designer, I’m finding that a significant portion of the answer of when to move forward is dependent on my own judgement and the constraints we are working under (certainly in a professional setting, constraint is a critical factor). One of my personal goals with this project is to take this design into digital prototyping. Given the amount of time left in the course and the strength and stability of my design at this phase, I’m comfortable leaving user testing behind (at least until the digital prototype is functional) and devoting my attention to the next phase.