From Systems to Experience: Shfiting Perspectives of Research

Before I came to AC4D, I spent six years working at a non-profit called the Connecticut Pre-Engineering Program (CPEP). At CPEP, I went from teaching classes to middle school students in the CPEP educational science and math summer program to managing that summer program. I eventually shifted to managing the CPEP organization while serving as Acting Director for five months. I realized recently that the systems mindset I developed over the years at CPEP has had shaped how I tend to make sense of things, and that systems approach has had some unwanted influences on how I’ve approached shaping my team’s design research.  I’ve changed my tune, I’m happy to say. This is the story of that change.

The revelation occurred one night during Research class. We had prepared our participatory design plan and were very satisfied with how the activities were laid out. Our focus was to understand what impact schools had on students’ awareness and agency in their food choices.

We were prepared to interview several school administrators who had the power to effect policy changes to improve students’ awareness and choice regarding food, such as through developing educational programs or integrating more food sampling into the cafeteria to habituate students to new and healthier foods. By interviewing these change-makers who had so much experience with students’ habits and reactions to new food, we presumed to develop a big picture of where there were opportunities to improve their system.

We had several participatory design activities set-up like pre-session homework questions to prime their thinking about student food choice and ideal solution activities that would help explore what a solution might feel like, e.g. simple to administer, community involvement, and so on. These activities would set-us up well to inform the development of policies or programs that would improve students’ awareness and agency regarding food consumption.

Here I was, looking to learn about the system so we could improve students’ experience. It turned out this was just a tad backward for the requirements of this research.

Class started, and we presented our ideas. The instructors patiently waited for us to complete our five-minute presentation. Then the curtain was pulled way back. To paraphrase them: “This is exactly how things are currently planned. Top-down. Why aren’t you interviewing students about their experience of food choice? Or else ask the administrators about how they experience food service planning.”

Our team was completely surprised. We had completely misinterpreted the nature of the process. We had tried to understand the system and then shape the experience, but this is not the proper way to approach it. The way I have come to understand it better is to frame the research around defining the solution and experience, and then to shape research around those definitions. Let me explain.

First, who owns the solution? Second, who’s having the experience? In the case of creating a solution for student’s awareness and food choice, it’s true, the administrators largely “own” the solution. They are the ones who would have to enact it, and they understand many of the constraints around creating the solution. They also have to “experience” the solution from the point of administering. But they aren’t the primary subjects who have the “experience” we are defining. The students are having the experience. What do they know about food? Where do they learn about it? What’s it like to see a new food item in the cafeteria? What choices do they feel they have around food?

My background as an administrator and educational program designer largely stilted my approach to this research. Teachers and administrators are (supposed) to be experts in education. They may know more about what is administratively possible or what is pedagogically researched and sound when developing lesson plans compared to a middle school student, but they are not the ones having the experience. If we were to develop a solution for a classroom, a key subject to interview would be students, and whatever the solution proposed may be, the teacher can own the solution.

After class, my team was faced with a challenge – we had spent almost 6 hours developing our participatory design research materials, and now we had strong criticism of the very basis for those materials. We had three interviews already lined up that were to take place within forty-eight hours, and it was now 10pm. What to do?

I’m happy to say that we changed our whole framework and question that night. Given our interviews were lined up with food service staff members, we decided to inquire about their experience rather than try to interview students about our prior research question. Our question was “what’s the experience like of developing school food menus?” We tested our the questions and material the next day with an executive chef, and it went swimmingly. The responses were rich, the stories were aplenty about the logistical, conceptual, and even aesthetic choices faced when developing menus for cafeterias that serve 5,000 students each day.

In summary, I’m slowly learning to adjust my approach to this kind of research. Even though I often took students’ and staff members’ input when developing the educational programs I managed, my headset is still to observe and learn about whole systems. As much as it is possible to learn about the system in which experiences take place, I am learning more and more to shift my lens for the lens to the singular experience of people in that system. I’m shifting my lens, pulling it into focus, and it’s making the research even more enjoyable.

Writing a narrative on User Research

This is the second project for theory class, and it feels like it’s the tenth. The pace at AC4D has been exhilarating and exhausting, and without a doubt it’s been educational. In theory class we’ve dived deep into several readings, and after each set of readings, we’ve been tasked to create a comic to illustrate the concepts learned. This has been a fantastic device as it forces us to read for understanding, synthesize the concepts across a range of authors, and exercise our creative and communication skills through developing an aesthetically pleasing and rationally sound argument through narrative. For the rest of this post, I’m going to unpack just what this all means and what I learned along the way. I hope it’s helpful to you.

The first set of readings or which we made a comic was due early September, and we read six pieces by six different authors’ all along the theme of a designer’s responsibility in society. The assignment was our first, and the learning curve was pretty steep, as Jon, our instructor, often chose to be more enigmatic about defining requirements rather than prescriptive about how to do the assignment. I for one failed miserably on the presentation given I talked about the process of designing the comic much more than I mentioned anything about theory or the narrative itself. This time around, I incorporated most all of the feedback and wish-I-hads into my project. More about that later.

Luckily, our second assignment was more or less the same as the first but with a different set of readings.

  1. Do the reading
  2. Write and illustrate a comic that both tells a narrative (replete with conflict and resoution) and ALSO communicates clearly each of the author’s positions.*
  3. Develop the final draft of the comic in Adobe Illustrator
  4. Post the comic with a blog post of about ~1000 words online.
  5. Present the comic in class for critique.

*In particular, Jon requested that we explore how each author would engage users in design research. Since the writings could be interpreted to cover a much wider range of topics than just design research, this bit of focusing criteria helped define a clearer choice of paths to take.)

As I said, this is essentially the same project as the first, and, in a school where there’s another new project every other day, I deeply appreciated the opportunity to iterate and improve on a particular project, especially one that utilizes so many important skills like theory analysis, drawing, illustration, visual communication, story-telling, presentation, and finally writing (this blog post right here in fact.)

The readings we did this time around included:

  • William Gaver – Cultural Probes and the Value of Uncertainty
  • Donald Norman – Technology First, Needs Last: The Research-Product Gulf
  • Jon Kolko – The Value of Synthesis in Driving Innovation
  • Liz Sanders & George Simmons – A Social Vision for Value co-creation in Design
  • Paul Dourish – What we talk about when we talk about context

The theme was more than anything about user research, though each writing touched on several other topics. Kolko’s article argued for gaining access to the boardroom for designers, Dourish discussed the depths of phenomenological and positivist world-views and their impact on contextual definition in computer science, and Norman spoke largely about innovation rather than user-research in detail. Nonetheless, the common thread was there, and it made for a rich set of articles.

In approaching the assignment, one of the greatest difficulties is both in synthesizing the author’s arguments so their perspectives can be lenses for the same conversation. The assignment is to develop a narrative, and what’s worse for a narrative than incoherence. Developing a sense for how these authors could dialogue together without talking about separate subjects was therefore the first challenge I set out to do in my own process.

To do this, I used a mix of mind-mapping and affinity diagramming. My version of affinity diagramming in this sense was to review my notes and highlight what I felt were the most important points each author made in their article. I then reviewed the different authors’ points side by side and started discerning themes and grouping.

On the other hand, once I had some themes written down, such as “Creativity of the user”, or “Role of innovation”, I wrote each author’s initials in a circle around that theme and wrote down each thing that author said about that point, if anything was said at all. This activity was more like mind-mapping or brainstorming.

After I had my author’s points synthesized and clarified around several themes, I tackled the question of narrative. This was somewhat easy for me. Ever since we started at AC4D, I’ve heard people talk about the challenges of convincing those with power to invest in user design research. With this in mind, I decided I would write out a narrative about a company that experienced challenges in the marketplace and decided to pursue user design research for the first time. In this scenario, a single person would be the advocate for design research, and they would both be inspired by the many arguments for it and also have to make the argument themselves to their board.

This approach both helped me learn the readings and also helped me prepare to make a case for user research investment in a boardroom.

Next, I needed the details of the narrative. After some brainstorming, I started mixing and matching ideas that I liked including educational settings like a school district (not a boardroom at all, I know, but it was a contender for winning narrative), and the corporate boardroom. I “compromised”, though really it feels like a total win, and decided to write about an education technology corporation. I personally love languages, and so I made it even more interesting for myself by making this company focused on language learning software.

Thus was born my comic strip.

Over the next 48 hours, I spent about 35 of them working on the story, deciding which drawing style I’d use, storyboarding, script-writing, hand-drawing, doing digital illustration in Adobe Illustrator, and finessing the format and composition. It’s been grueling.

That being said, I’m very proud of my work. I know that it could be better. It can always be better. But I invested so much effort, and I have pretty much only worked and not slept for the last 24 hours.

I love the theme of language ed tech that came to the surface. I love that I worked so hard to make my pretty amateur drawings at least a bit more polished. I love that there’s so much detail and there’s even a couple plot twists in such a short story. I love the character, Sarah, who came to the surface – she really seems striking to me. She’s courageous, compassionate, and dedicated, and she embodies many of the values I admire. It’s no surprise of course given she’s my creation, but then again, sometimes we don’t let those values come to the surface, and luckily they did here.

I learned a lot from this project. I learned about great theory related to user research, co-design, using cultural probes, and the intersections and implications of philosophical world-views in computer science. I’m well over 1000 words now, so if you want to know about those topics, you’ll need to look those articles up yourself, (or read my comic.) What I perhaps have valued the most from this project was the learning developed through rigorous creative work. Creative works requires the most varied and highest level of skills and cognitive skills, and it’s been taxing to say the least to fit about 100 pages of dense theory into a comic strip narrative. Nonetheless, I look forward to the next time.

Note: I set up my comic below as a gallery. Click the first image, and it will open in a black box on the screen. There are navigation buttons that appear on the bottom of the slide when you hover over the bottom. Use those navigation buttons to scroll from page to page rather than going in and out of the media files. Thank you for reading.

A glimpse inside the kitchen

From the outside, the building is unassuming – brick, short, and with a simple, flat facade. As we walked through the front door, however, we immediately heard the chatter of teachers in the school cafe and the clanking of dishes, the whirring of washers, and the banging of ovens in the cafeteria kitchen. We began our first design research project for the AC4D design program just a few weeks ago, and our focus is to learn about the factors and actors that influence school food service. The day we walked into that school was the day I really got a taste of just how little I knew about the world I was about to design for. It’s now clear to me just how important contextual research can be to paint a more rich picture of that world and to inform our design.

Up until that point, our team had lived in a world of stickies, conversation, laptops, and the interior of the AC4D classrooms, with occasional breaks for tacos nearby. We talked in class about the importance of user interviews and ethnography to gain a fuller perspective of our research topic and to develop greater empathy with the people we would design for, and as we delved into scoping out our research topic, it felt alive with curiosity, questions, confusion, personalities, and debate.

But in a way, we didn’t really known anything about our topic until we arrived to the first school cafeteria.

We arrived at the school, and the noise, sights, and people who we had previously only talked about came to life. In fact, most of what we saw and heard we hadn’t even talked about or imagined. The teachers’ separate cafe and meeting space, distinct and shielded from the din of students in the cafeteria; the cafeteria workers’ conversations in Spanish; the very small staff that ran a huge cafeteria operation – these were all unforeseen details of a world that kept unfolding before us, proving the topic more complex with every new glimpse inside it.

This was clearly not the world I imagined. Indeed, the world is not what we think.

I think of a school cafeteria, and images from sitcoms immediately come to mind. Or maybe images from my own experience attending school come to mind. Or else images sparked by news articles and stories about school cafeterias come to mind. When I tried to imagine the world of school food service prior to this visit, these images were likely feeding my imagination. But all these sets of images are blurred by vagaries of my own memory, and more importantly they represent superficial impressions of that domain as a whole.

During our first visit, when we met our participant, she didn’t look anything like what I imagined a school food service director would look like. What’s more, during the interview, instead of viewing her job as a relentless barrage of administrative and regulatory tasks as I had projected, she painted a quite different picture for us. When we asked what feedback she got from parents, her response was that yes, she does get feedback, requests, and complaints – and she completely understands parents’ earnestness to communicate with her. After all, she’s responsible for feeding thousands of other people’s children, and it’s an honor and a huge responsibility.

How did I never think of it this way? This food service director has a strong emotional connection to her work, and an immense responsibility and connection to that charge. But for some reason, my mind was busy with looking at the emotions that may arise due to faults or snags in the system, administration, or logistics. This insight into how she sees her role and her responsibilities could be central to a design solution. It could inspire solutions that create more caring, devoted, and responsible school nutrition services for thousands more children. If I had projected my lens on her role, making administrative tasks easier, that would likely have inspired a much different and likely less effective, energized solution.

Over and over, throughout that first visit, the world I only faintly had imagined grew bigger, more complex, more rich, more connected, and, in a way, more incomplete. Our minds want to make things whole, so we imagine we see the whole. This illusion of wholeness, of the completeness of our knowledge, however, can be the undoing of the solutions we strive to create. Once we assume we know the whole picture, we stop learning and developing an even more complete picture that can be used to inform the design process. Contextual design is a process that not only gives designers a better picture of the world they’ll design for but also reminds them of how incomplete their ideas are. It is certain that our perspectives are far, far from complete. To put another way, every meal has a story, and if you want to know what’s on your plate, it’s best to look inside the kitchen and ask the chef – but even then, it’s just a glimpse.

Design on Trial: Designer’s Responsibility for their work

This story is about the responsibility of designers. It isn’t common to hear of a designer going on trial because his or her design caused harm, but through this course and the associated readings about design and it’s responsibility to society, it seems that it is critical to reflect on designer’s responsibility. Perhaps one day designers will go on trial as depicted in this story.

This project was truly a challenging one that helped me develop multiple skills. The challenge was to present six authors’ viewpoints that we had read in class in a comic-strip type story. From learning to use Adobe Illustrator to publishing in WordPress, the technical skills we needed to learn were useful and also full of unexpected troubleshooting.

One of the most difficult things to do in this assignment however was to make almost one-hundred pages of mostly dense and abstract theory into a story that is grounded in a concrete narrative.

Given responsibility was one of the key themes in class so far, however, a trial scenario seemed to be a good way to set the theory in narrative. Over the course of the trial, the authors present as expert witnesses and debate the fictitious designers’ project and it’s dire fall-out for Zambia. This format was enjoyable to let the designer’s snafus and narrative unfold through the court case.

The designers make many mistakes, but whether or not they are responsible is a very complex and difficult decision to make. For just that reason, the reader of the comic is left to make up his or her own mind. I hope it helps readers think deeply about designers’ responsibility in society and make up their own mind about whether they should be held accountable or not.