Iteration and User Testing: Round 3

Taking a design idea from low fidelity to functional prototype requires a significant amount of attention to detail and each decision has implications for the user.  In our Rapid Ideation and Creative Problem Solving course we’re learning methods to help us address the inherent difficulty in this process by leveraging user testing and iterative redesign.  I’m quickly discovering that this makes the design process less arduous and most robust.  Major flaws are quickly revealed and the burden of addressing every detail in each pass is replaced by an attention to the most essential elements first.  What emerges for a designer that embraces this aggressive, fun approach is often more simple and elegant than what they would design in isolation.

Our class is specifically focused on redesigning a thermostat interface.  I’m currently straddling my second than third iterations of the design and hoping to move into an interactive prototyping phase soon.  You can view my initial concept models as well as my first design iteration and second design iteration for more detail on how I arrived at my current design.

In my last round of testing I discovered a significant hiccup in my design for some of the people who tried to use my prototype from last week (pictured below).


Several people really struggled with the use of the outer dial on the device as a means of navigation.  After problem solving with an experienced interaction designer, I tried to implement a new version of my interface that will hopefully avoid the same issues.

One issue that user’s had is that the outer dial didn’t translate as a way to control a linear band like the one shown above.

The other issue is that the notion that some users were perplexed and had differing interpretations of the navigation menu.  Some users though that turning the dial would control the triangular arrow while other though that the menu items themselves would rotate (like and oven dial).

I tried to change these interactions the map to a more consistent mental model and that involved a significant change in a lot of my design.

First, I created a circular band (to replace the linear one) at the top of the display (shown above).  I believe this will reference using the dial to adjust the temperature more directly.

This called for a new way of articulating a changing desired temperature.  I handled this by introducing an enlarged dialog that follows the selector as the user turns the dial (show above).

Once the user has confirmed their new desired temperature (by pressing the device or through inactivity) the dialog shrinks and the display indicated how long the according behavior will be in place (as shown above).

Meanwhile, I addressed the navigation issues by using a consistent model for how the dial controls things. If the user rotated the dial (above) to the right then they would see this:

Notice that the display is also dimmed to indicate that the user is still in navigation.  Once they press the device, the device will return to its normal brightness (see below) and they will be able to adjust the setting in COMFORT RANGE AT HOME display.

Off for a another round of testing.  If this round is more consistent, I’ll probably start trying to push my prototype into flash soon.




Young Designers Should Start Companies

Designers should start a companies.

Fact is, junior designers have no influence in a corporation.  I don’t want to be a part of that.

You can take the traditional corporate path, or you can start a company.  It’s different, and different is scary, but my design education prepared me for iteration and solving difficult problems.

This isn’t a walk in the park.  Making decisions, project managing myself, and taking responsibility isn’t easy.  I think many people are afraid of that responsibility.  I know it because I live it everyday, as I’m trying to grow both my design and business skills through bringing a product to market.  It’s unnerving, but exciting.  I prefer excitement to a structured trajectory as long as it’s economically feasible.

And I prefer it to the the cost of most graduate school education options.  You aren’t going to make any money at the start of an enterprise.  But  let’s compare that to MBA education.  What if you just paid yourself 60K a year (well, maybe in time equivalent) instead of putting it into school coffers?  And if you’re smart and have saved some money, why not?

I may or may not make it at this entrepreneurship game this time around.  Experience from my program indicates that those who start their own companies are likely to fail in the short run.  This shouldn’t be surprising.  The best business leaders in the world fail for a myriad of reasons.  But the worst case isn’t bad.  I’ve seen my fellow business designers (“biz des”ers?) skip the corporate crawl.  One individual who just 2 years ago entered IxD is now a UX director at a local Austin company.  That doesn’t happen out of grad school typically.  It happens after running your own business.

Talk about showing grit to your potential self, supporters, and users.  Pay it forward now by starting something that’s your own.

Innovation and Statistical Significance

During our fourth day of orientation, students at AC4D started learning about the process of extracting insights from data. Inevitably, the topic of statistical significance was discussed. Surely we can’t trust these insights? We only talked to five or six people; these people weren’t randomly selected, and they might represent an anomalous point of view.

If I had a dollar for every time I’ve had this conversation, I would have enough money to change the course of K-12 education to be less focused on analytical thinking. Our efforts to quantify, track, and measure everything have left us in a place where creative insight and provocative thought is almost automatically tempered by a desire for proof. Roger Martin describes that “the enemy of innovation is the phrase ‘prove it’“, and he’s right. During the process of insight extraction and envisioning the future, ideas are fragile, and if you work in a culture driven by data, it’s likely that your idea won’t ever get a chance to grow past their seedling stage.

Simply, statistical significance is irrelevant during research and early stages of innovation brainstorming, because the goal during insight extraction is provocation, not prediction. Designers who are synthesizing research data aren’t trying to make mass generalizations based on what they learned from a few. Instead, they are trying to provoke new realities and look at the world in new ways. It’s a playful process, not a scientific process. And it happens best when it’s separated entirely from a conversation of market forces.

Will this idea scale? Who knows? There’s plenty of time to push ideas through the analytical bean-counting reality of bringing a product to market. During research and synthesis, focus on a local view of product-market fit, one that punts on topics of scale and instead emphasizes emotional value. Give your little happy seedlings of innovation time to grow before you spray Excel scale models all over them. *

* I recognize that the metaphor is terrible. Still, I had a funny mental image of a spreadsheet being sprayed all over my garden, so I went with it. 

Abductive inference and sense-making

In our latest batch of readings for our theory class, I was particular interested in deconstructing a lecture from Charles Pierce. He is considered “the father of pragmatism.” Given that he is a philosopher and logician, his lecture, “The Three Cotary Positions,” is particularly thick, and I found difficult to parse: which made it a great candidate for using diagrams to make sense of it! This brought to mind another reading, “Organizing and the Process of Sensemaking,” by Karl E. Weick, Kathleen M. Sutcliffe, and David Obstfeld. So in this diagram, I’ve woven together both concepts to describe a process and relationship between abductive inference, synthesis, hypothesis testing, and sensemaking.

Click to download a PDF version

Connecting Design Research to Value

There’s a simple way to illustrate the value proposition of a new company, and I’ve found it to be extremely effective in communicating the worth of a hypothetical new product or service.

First, introduce an actual user that you’ve spoken with. If you are using presentation software, like Powerpoint, use a full-screen image of the person doing their job. This suggests that you’ve spent time with a potential user, and it immediately humanizes your intent: it indicates that you are presenting design-led innovation, as opposed to technological or business-led innovation.

Describe the person’s main want, need, or desire. This is sometimes called a “pain point”, but I feel that the word “pain” is too simplistic (the language I’ve used – want, need, desire – is probably too simplistic as well) because this is often subtly aspirational. Illustrate that you both understand and empathize with the person by emphasizing the emotional result of this need not being met.

Use their words. Quote the user, verbatim, in order to substantiate the need. If you are using slides, I’ve found it extremely effective to overlay the quote in REALLY BIG LETTERS on top of the user.

Repeat for two or three users. Show that you’ve spoken with several users and identified a running theme, a pattern.

Summarize your synthesis. Using a single slide, show the users again, and illustrate the high level summary of your interpretation of your research. This is where you show an inferential leap: where you combine empathetic research data, and build upon it, to produce insights. I’ve found it useful to show each user again, summarize their quote, and then show my interpretation directly below it.

Identify the implications of your synthesis. Using a slide per insight (no more than three), explain what the implications of your insight are on a potential new system or service. At this point, you are identifying new constraints: you are describing how you are artificially constraining a blank canvas of new ideas, in order to suggest a new and valuable service.

Introduce the product or service. Use the widely used formula: We help [your most promising prospects] that [need help with the pressing concern you address] succeed by [providing the material improvement you will deliver].

From here, alternative approaches work, depending on the audience. If you are presenting to investors, you might show how the service works to generate revenue, and then transition into a discussion of financials. If you are presenting to a technical audience, this might be an opportune time to introduce a “how it works” diagram, emphasizing the technological stack and architecture.

Using this style of presentation works because it gives your audience a point of reference, a place from which to judge your design and idea. It helps them see the world from a different perspective. And it offers a rationalization for your product or service, but in human terms. It doesn’t try to prove the giant potential of your market, which investors see through quickly, and it doesn’t claim a massive technical innovation, which technologists are implicitly skeptical of. It’s a designerly way of showing value.

Abductive Reasoning in Airport Security and Profiling

When we speak of an inference, we usually imply a leap – a jump from one point to another. Between these points is a gap, and the inference is the bridge between the gap. In typical forms of (inductive) logic, the gap is between fact and observation:

[a fact:] All meowing cats are hungry.
[an observation:] My cat is meowing.
[an inference:] Therefore, it is hungry.

But we mean another kind of leap when we speak of inference in innovation, and we commonly refer to this type of inference as being less logical: as an emotional, intuitive, or feeling-based inference. Consider:

[a fact:] Cats sometimes meow when they are hungry.
[a fact:] Cats sometimes meow when they are hurt.
[a fact:] Cats sometimes meow when they are happy.
[a fact:] It’s 6am.
[a fact:] I usually feed my cats at 6am.
[an observation:] My cat is meowing.

These are three potential valid inferences:

[an inference:] My cat is meowing because it is hungry.
[an inference:] My cat is meowing because it is happy.
[an inference:] My cat is meowing because it is hurt.

All three inferences are valid, because they follow from the facts. But one is much more likely than the others to be true. This is the bread and butter of a design-based argument for an innovative new idea: the ability to identify a valid inference based on constraints, and more importantly, to then treat the valid as fact – and build upon it. This is abductive logic, coined and described by Charles Peirce as a form of “Retroduction.” As he describes, and I quote in full:

“The inquiry begins with pondering these phenomena in all their aspects, in the search of some point of view whence the wonder shall be resolved. At length a conjecture arises that furnishes a possible Explanation, by which I mean a syllogism exhibiting the surprising fact as necessarily consequent upon the circumstances of its occurrence together with the truth of the credible conjecture, as premisses. On account of this Explanation, the inquirer is led to regard his conjecture, or hypothesis, with favour. As I phrase it, he provisionally holds it to be “Plausible”; this acceptance ranges in different cases — and reasonably so — from a mere expression of it in the interrogative mood, as a question meriting attention and reply, up through all appraisals of Plausibility, to uncontrollable inclination to believe. The whole series of mental performances between the notice of the wonderful phenomenon and the acceptance of the hypothesis, during which the usually docile understanding seems to hold the bit between its teeth and to have us at its mercy — the search for pertinent circumstances and the laying hold of them, sometimes without our cognisance, the scrutiny of them, the dark labouring, the bursting out of the startling conjecture, the remarking of its smooth fitting to the anomaly, as it is turned back and forth like a key in a lock, and the final estimation of its Plausibility, I reckon as composing the First Stage of Inquiry. Its characteristic formula of reasoning I term Retroduction, i.e. reasoning from consequent to antecedent. In one respect the designation seems inappropriate; for in most instances where conjecture mounts the high peaks of Plausibility — and is really most worthy of confidence — the inquirer is unable definitely to formulate just what the explained wonder is; or can only do so in the light of the hypothesis. In short, it is a form of Argument rather than of Argumentation.

Retroduction does not afford security. The hypothesis must be tested.

This testing, to be logically valid, must honestly start, not as Retroduction starts, with scrutiny of the phenomena, but with examination of the hypothesis, and a muster of all sorts of conditional experiential consequences which would follow from its truth. This constitutes the Second Stage of Inquiry. For its characteristic form of reasoning our language has, for two centuries, been happily provided with the name Deduction.”

Built into an abduction is the possibility of error – a risk of making a leap that is valid, but false. I could, based on my assessment of the situation, rush my cat to the vet only to find out she’s hungry. More likely, and more unfortunately, I might feed my cat and never take her to the vet, only to find out something’s wrong. The signals I receive help me build an assessment of the situation. When I act on that assessment, there is risk, and when I’m wrong, there are consequences. These ideas of inference, risk, and consequences are at the heart of innovation. An innovation is, simplistically, the result of abductive reasoning; I wrote about this extensively in my second book.

But I don’t take my cat to the vet every morning. That’s because, while my cat may be meowing because something’s wrong, and there will be terrible consequences if I fail to take her to the vet and something’s wrong, the practical burden of acting as if all inferences are true is too large. What’s more, there are other signals at play that help me build a stronger case, intuitively and automatically, that the cat is simply hungry: she walks around, and makes eye contact, and there have been no other visible indications of problems over the last few days, and her coat is shiny, and so-on. Peirce’s word choice was purposeful (my emphasis added): “the search for pertinent circumstances and the laying hold of them, sometimes without our cognisance, the scrutiny of them, the dark labouring, the bursting out of the startling conjecture, the remarking of its smooth fitting to the anomaly, as it is turned back and forth like a key in a lock, and the final estimation of its Plausibility.” The process of identifying the abductive insight is long, winding, thoughtful, reflective, and murky.

My reflection on inference this morning is based on the thread between Sam Harris and Bruce Schneier on airport security. (If you missed it, start here, then read this, and finally, this)

Profiling is a form of abductive reasoning, and in airports, it’s based on facts, observations, and inferences like this:

[a fact:] Some terrorists are Muslims
[a fact:] Some Muslims wear unique forms of clothing, like the Hijab
[a fact:] Some Muslims have dark skin

[an observation:] That person, who has dark skin and is about to enter airport security, is wearing a Hijab.

[an inference:] That person is going to visit her relatives
[an inference:] That person is going to blow up a plane

Like the example of my cat meowing, both inferences could be true. And like the example of my cat meowing, one inference is much more likely to be true. And, like the example of my cat meowing, the consequences of failing to act based on one of the inferences could be catastrophic. And, like the example of my cat meowing, we shouldn’t act on it simply based on the potential for a catastrophe.

Harris claims that “We should profile Muslims, or anyone who looks like he or she could conceivably be Muslim, and we should be honest about it. And, again, I wouldn’t put someone who looks like me entirely outside the bull’s-eye… But there are people who do not stand a chance of being jihadists, and TSA screeners can know this at a glance… At a minimum, wouldn’t they want a system that anti-profiles—applying the minimum of attention to people who obviously pose no threat?”

Harris is actually making two claims at once:

  • We should profile people who “look Muslim”
  • We should profile people who “do not look Muslim”

Both are complex ideas; he never unpacks what it means to “look Muslim” (And his assumptions are probably wrong). But while that’s a flaw in his argument, I have two larger problems with his jump to profiling, based on how I understand inference-based abductive reasoning to work.

First, I don’t think you can perform an abductive leap as quickly as would be necessary, in the context of an airport, to spot a terrorist. Abductive reasoning is based on multiple signals coming together in new and interesting ways, and it takes time for the brain to process those signals. Based on my own experiences, I would argue that the “bursting out of startling conjecture” that Peirce describes can’t actually happen without sleep, where neural pathways are reconfigured and rearranged. I think it’s physiologically impossible to make the necessary leaps to do this work.

But more importantly, abductive reasoning is inherently a risk-based process, where the risk is that you may make an incorrect inference, and that incorrect inference has consequences. The negative consequences of false-profiling Harris describes are nearly all social, but are enormous. They will further alienate a culture that is already vehemently angry with the western way of life and culture, and these activities will likely produce more of the behavior they are intended to stop.  Because each time the leap is wrong (which will be nearly all of the time), the consequences will be reified.

We don’t have a strong understanding of how this type of inference works. We’re starting to understand more about the plasticity of the brain and the way snap judgments work, but we’re years away from truly knowing how new knowledge is produced. Consider the crap-shoot that is new product development, and our collective lack of ability to introduce innovations into the world with any repeatable form of success. I’m highly skeptical of building national policy on top of such a way of thinking.