Innovation and Statistical Significance

During our fourth day of orientation, students at AC4D started learning about the process of extracting insights from data. Inevitably, the topic of statistical significance was discussed. Surely we can’t trust these insights? We only talked to five or six people; these people weren’t randomly selected, and they might represent an anomalous point of view.

If I had a dollar for every time I’ve had this conversation, I would have enough money to change the course of K-12 education to be less focused on analytical thinking. Our efforts to quantify, track, and measure everything have left us in a place where creative insight and provocative thought is almost automatically tempered by a desire for proof. Roger Martin describes that “the enemy of innovation is the phrase ‘prove it’“, and he’s right. During the process of insight extraction and envisioning the future, ideas are fragile, and if you work in a culture driven by data, it’s likely that your idea won’t ever get a chance to grow past their seedling stage.

Simply, statistical significance is irrelevant during research and early stages of innovation brainstorming, because the goal during insight extraction is provocation, not prediction. Designers who are synthesizing research data aren’t trying to make mass generalizations based on what they learned from a few. Instead, they are trying to provoke new realities and look at the world in new ways. It’s a playful process, not a scientific process. And it happens best when it’s separated entirely from a conversation of market forces.

Will this idea scale? Who knows? There’s plenty of time to push ideas through the analytical bean-counting reality of bringing a product to market. During research and synthesis, focus on a local view of product-market fit, one that punts on topics of scale and instead emphasizes emotional value. Give your little happy seedlings of innovation time to grow before you spray Excel scale models all over them. *

* I recognize that the metaphor is terrible. Still, I had a funny mental image of a spreadsheet being sprayed all over my garden, so I went with it.