This week we got another usability study in the books by testing the most recent iteration of our signup and onboarding flow. Thanks to some help from our design operations team I was able to pull it off without a hitch. All I had to do was fill in a request, slightly massage a template script, and then show up for the sessions. Our recruitment coordinator, the amazing Lilibet Greig, did the rest by finding participants, screening them, and then following up with them to make sure they showed up for the study.
Running the test
Over the course of a week, I worked with two other designers to interview six participants and watched them build a website with WordPress.com. Each key segment of our customer base was represented and their experience using site builders ranged from novice to professional.
Starting from our marketing home page, they were asked to verbalize their thoughts as they made their way through the process. While one of us spoke with and guided the participants, the others documented their observations and quotes as individual cards on our virtual Mural board. The board was divided into sections representing each screen in our flow and we used different colour “stickies” for each participant.
Synthesizing the results
Once we completed the sessions, I made sure to go through all the recordings, including the ones I moderated, and added my notes to the board. It might seem redundant for multiple people to take notes but I think it’s important to bring multiple perspectives from different view points because we all perceive things differently.
We gathered a great deal of raw data from this study and I won’t lie when I say it was kind of intimidating to think how long it would take to comb through it all but I eventually got through it thanks to the colour coding and organization of the board. I read every single note and grouped similar items to identify patterns which lead to insights that I used to put a report together along with my recommendations.
Creating a report
The final report turned out to be fairly long due to the amount of ground we covered in the study. I therefore summarized the top findings and recommendations on the front page to make it easy to digest. I made sure to include direct quotes and video snippets though out the rest of the report to back up the insights because nothing beats seeing things with your own eyes.
Now that we have our list of recommendations, I’m going to work with my product and engineering partners to prioritize them using the ICE method.
The outcome
Doing usability studies, and customer interviews, are always exciting for me because not only can you uncover problems with your design but you can also be inspired with new ideas. This study was no different — we uncovered some issues in our flow and by means of observing people use our product, I come up with some new directions to explore.
Automattic Design 