Over the years of conducting user tests my team has used and iterated on our own usability test script. We’ve refined the structure and language, optimizing for objective, unbiased user insights.

When starting a new test we’re now able to mobilize pretty quickly, building off our past efforts. A big part of the efficiency is creating the test script with the template we’ve put together.

In the spirit of open source I’m proud to share our Usability Test Script for use by anyone who can find it useful!

---

Open the Usability Test Script

Contributors: Mike Shelton, Brie Anne Demkiw, Dan Hauk, Ran Hartstein, Ben Dwyer, Filippo Di Trapani

---

Guide to creating your test script

The script is ready to be used, just needs a few test specific pieces from you. However, I’ll share some tips and experience that helped us to create the template, as well as some helpful info for customizing for your needs.

A good script generally consists of these 6 components:

1. Intro
2. Pre-test interview
   
3. Scenario
4. Tasks
5. Follow-up questions
6. Post-test survey

Let’s look at what each component is comprised of.

The Intro

When you begin a test you want to set the mood for the session. Make the participant feel comfortable. Explain how the test will work, but don’t give away any details that could bias their actions or opinions. Reassure them that they’re not being tested. Cover any other business items before diving into the test.

Example:

Hi _________! How are you doing today?

Thanks for offering to help us out, we really appreciate it!

I’m Mike, a designer for WordPress.com and I’m joined by a couple other folks who will be tagging along today with us.

Introductions

We work for Automattic, the parent company to WordPress.com and we’re working on making it easier to find, purchase, and manage domains.

So today you’ll be trying out an early prototype of a website. We’ll ask you to complete a few tasks using the site. We will be following along with you observing your actions to see how easy or hard the site is to use.

Don’t worry though, we aren’t testing you; we’re testing the site. There is no wrong answer, and there’s no way to make a mistake. All of your thoughts and feedback will be very valuable information as we look to improve our product. On that note, we want you to think out loud as you view and interact with the website. We want to hear what’s going through your mind as you complete the tasks.

I should also mention that no information you enter on the website is being saved, so no need to worry about that. In fact, we’re not even concerned if the information you give is accurate; we just want to see how you use the website.

We are recording this session for our reference as we work to improve the website. The recordings will only be seen by Automattic employees working on this project.

Do you have any questions for me so far?

Pre-test interview

Start off by asking a few interview questions to help them get warmed up to sharing their thoughts. Be careful not to ask questions that might reveal too much about your intentions though. Use pre-test questions to understand the participant but don’t plan on using them to collect any useful quantitative data –  a separate survey or interview with a separate cohort will be better suited for that.

Example:

1. What kind of work do you do? / Tell us about your business.

2. How long have you been in business?

3. What does your average day working on the business look like?

4. What online tools do you use most frequently in running your business?

Scenario

Next, you need to set the scene. You want the participant to assume this frame of mind for the test so make it easy to follow and relatable. It’s also a good idea to have the scenario written somewhere for the participant to refer to as needed.

Note: If you’re testing a prototype or something not complete, your scenario will likely need to be more prescriptive. You’ll need to create an imagined situation  that aligns with the functionality of your prototype.

Example:

Imagine that you own a small business called Java Joe’s, a coffee shop in San Francisco that specializes in cold brew. You’ve been wanting to grow your business, and recently started a website on WordPress.com to attract people to your location. The website is great, but you’ve been checking the stats and noticed it hasn’t been getting a lot of traffic yet.

Tasks

The tasks are the essential element to the test that the most insights will be revealed through. The tasks for each test will differ, also the number of tasks too. Some tests will have only one task, some will have 10. Though a good rule of thumb is the fewer the tasks the more likely you’re testing the most natural, unprompted experience you can (in a usability test). If you break up the test into very specific tasks you are likely not truly testing someone’s ability to complete those tasks on their own.

Examples:

You’ve just logged into WordPress.com to look at your site’s stats page. Use the tools made available to you to help improve the traffic your site is getting.

Find and purchase a domain for your blog. You aren’t sure exactly what you want but you’ve been thinking about something like “The Roaming Forks”.

Ok, now that you have purchased a domain for your new blog, you want get it setup. Remember that this is a new blog you haven’t created yet

Follow-up questions

After each task or at a natural pause in the test, plan to ask follow-up questions. Try not to interrupt the participant during a task. You can ask situational questions that were prompted by the participant’s actions but also plan to have some scripted questions you would like answered by all participants.

Example:

1. Tell us why you chose _________.
2. Did you feel like you had enough information to make a decision about which service to choose for your new blog?

Post-test survey

As prescribed above, asking follow-up questions is good practice. However, we’ve found that when asked to rate an experience or provide constructive criticism during the test session, participants tend to give favorable answers because they don’t want to hurt your feelings. It’s best to ask opinionated experiential questions in a post-test survey that you send to the participant after the test.

Pro tip: To ensure participants complete the follow-up survey and in a timely manner try including a question in the survey relevant to claiming their incentive you offered them for completing the test.

Example:

1. What are two things about this site that you didn’t like?
2. What are two things about this site that you really liked?
3. On a scale of 0-10. How likely are you to recommend this site to a friend or colleague who’s looking to register a domain. 0 being “Not at all likely” and 10 being “Very Likely”?
4. Which would you prefer: A $50 Amazon gift card or a 1 year WordPress.com Premium subscription?

Enjoy and contribute

Please feel free to use this script as you wish. This is a living document and we will continue to iterate over time. Likewise, if you have any suggestions for improvements please make a comment in the Google doc. Enjoy!

Photo credit: Luca Sartoni