Evaluation my myAT&T Mobile Redesign

In previous blog posts I have detailed how we the class of 2017 has been tasked with redesigning the current myAT&T experience. We conducted research to conclude the top pain points/features we were to incorporate into our design, concept mapping to gain a high level understanding of the space. We then applied divergent thinking to focus on the experience without being distracted from the granular details. Next was to start manifesting our ideas by sketching out screens to get our ideas into a more tangible form, followed by increasing the fidelity by digitizing our sketches into wireframes.
The next step in the process is to perform evaluation through three primary methodologies: think aloud protocol, cognitive walkthrough, and heuristic evaluation. After spending so much time in the designs it’s easy to become tunnel visioned and miss opportunities to make your designed experience better for the end user. This potentially limiting viewpoint affords the needs to conduct the previously mentioned evaluative methodologies.
The first methodology I conducted is “think aloud protocol,” which evaluates the usability of my designs by encouraging a user to think out loud as they use your product or service with specific tasks to complete. The intention is to understand how users will navigate my designs, and more importantly the thoughts behind the what and why the user is performing certain actions. There may be a resistance to conducting this exercise due to the assumption that verbalizing internal thoughts will interrupt the task at hand. Experiments were conducted to validate this methodology and found that there is no affect on thought sequences, as long as there is no introspection. This is primarily why the only thing a facilitator of this exercise can say is, “please keep talking.” This avoids any introspection that wouldn’t normally exist within the user.

I created an InVision prototype and recorded the user’s comments, reactions, and  screen activity.

Overall, I conducted two rounds of think aloud protocol. One round with five users, and one round with three users. Between the two rounds I iterated on my designs applying feedback, then conducted the second round to validate whether or not my new designs were successful. New opportunities were presented during the second round as well.
Next, a “cognitive walkthrough” was performed. Cognitive walkthrough is a methodology used for evaluating the learnability of a product based on a theory of problem solving in unfamiliar situations. Think if it as examining the designs from the mentality of a user who had never seen the product before. Although this was not done with actual users, it helps frame the mind to look at the system from the point of view of creating something that doesn’t need any training to use. A cognitive walkthrough is performed by focusing on the primary tasks the user would want/need to perform within the app. These tasks include:

1. Compare your current usage to other data plans, then change your data plan accordingly.
2. Upgrade your current device to the new iPhone 7.
3. Suspend your phone because it was stolen while you were on vacation.
4. Process a insurance claim because you cracked your screen.
5. Make a payment on your current wireless bill and enroll in auto-pay.
6. Update your account’s password.

While examine each task screen by screen I asked four questions to arrive at opportunities to improve the learnability of the design. The questions include:

1. Will the user try to achieve the right effect?
2. Will the user notice that the correct action is available?
3. Will the user associate the correct action with the effect that user is trying to achieve?
4. If the correct action is performed, will the user see that progress is being made towards a goal?

If any of the previous questions show a problem, I then logged that problem into a spreadsheet with a unique identifier, unique screen ID (labeled prior to examination), problem description, severity (1-5, 5 is high), frequency (1-5, 5 is common), and a proposed solution. This makes it easy to identify specific problems relating to specific screens and prioritize opportunities within the design that would have the most significant impact to the overall experience.

Cognitive Walkthrough Thumbnail


The last methodology used during this process is called “heuristic evaluation.” The purpose of this exercise is to compare an interface to an established list of heuristics, or best practices, to identify usability problems. These 10 heuristics have been identified by Jakob Nelson in collaboration with Rolf Molich in 1990 and is generally deemed by others to include good principles to follow. Even though they were established 20 years ago, they still hold true to today’s usability standards. Heuristic evaluation is arguably more detail oriented as you identify each individual control on every screen and compare it to these heuristics, which include:

1. Visibility of system status
2. Match between system and the real world
3. User control and freedom
4. Consistency and standards
5. Error prevention
6. Recognition rather than recall
7. Flexibility and efficiency of use
8. Aesthetic and minimalist design
9. Help users recognize, diagnose, and recover from errors
10. Help and documentation

To capture these areas of opportunity a very similar spreadsheet is used as the one used in cognitive walkthrough. The difference being it helping to capture which of the 10 heuristics a specific control violates to be able to easily refer back to. The spreadsheet serves the same purpose in prioritizing efforts where would have the largest impact throughout the system.

Heuristic Evaluation Thumbnail


After the three evaluation methods were conducted, I then took the culmination of my findings to be able to decide on which changes could provide a significant improvement to the overall experience. Although there was plenty room for improvement, I chose the following three opportunities.
The first improvement I decided to make was definitely the most simple and easiest to implement, but still has very beneficial implications. When users were asked to both suspend a device and submit an insurance claim, their first tendency was to navigate to the account tab. When there were no options for device management they paused and were confused on what to do next. Eventually they realized that tapping the device tab would give them the necessary options. To solve for this I decided to simply add a manage devices options within the account tab. This will then redirect the user to the devices tab.

First Improvement

The second improvement I decided to to take advantage of is also in the devices section. Before I conducted evaluation I placed a CTA (call to action, or button) below the currently selected device which would then drill down to see more device options. There are a couple of opportunities here. For one, the device information at the top of the second screen shows a number of redundant information and doesn’t add any extra value the user wasn’t already exposed to. In addition to this, the current phrasing of “view device details” could be misinterpreted to mean device hardware information. To solve for both of the issues I decided to consolidate the two screen into a single screen. I believe this solves these problems by eliminating the confusing phrasing and immediately providing the options the user may want.

Second Improvement

The third opportunity I decided to work through was working with the visualization of data usage for the month. There were also a few opportunities within this space. One of which was simplifying the current billing cycle information as well as the data usage information into a single chart. Users were having a difficult time processing the way I displayed the information within a bar. This placement also was difficult to compare, which was another problem. Being able to compare your current usage wasn’t as clear as I’d like it to be. Users were having to take time to process what was on the screen rather than scanning the screen. Lastly, I noticed that users were trying to tap on each individual data plan bar to update their data plan. To combat this I added a button for each plan. I think there is still more opportunity to make this specific screen even more user-friendly which I will continue to explore.

Third Improvement

Moving forward I will continue to work on these key opportunities, but also work on a number of smaller details that will make the overall experience more user-friendly. As a class, we will also be meeting with a couple of professional developer to gain a better insight into feasibility of our ideas and help create a plan of prioritization.

If you would like to see my presentation, you may view it here.