Recruiting participants for remote user tests

I recently attended my first remote design sprint at Automattic. It ran over a two week period and  was attended by fellow designers, developers, data scientists, happiness engineers, and marketers. Together we created two prototypes and tested them out with real customers. I found the experience to be both rewarding and inspiring. We came up with a lot of great ideas and validated them too. It was also really nice collaborating with a number of people across the company that I don’t usually get to work with.  

One of my major contributions to the sprint was recruiting real customers for the prototype test. The job was a whole lot easier thanks to an internal post written by my designer colleague Mike Shelton which also took part in the design sprint. He shared his experience recruiting customers from a previous test and also included the templates he created for people like myself to use. With that post, I had everything I needed once we determined exactly which type of customers we wanted to test.

Finding participants

Before the sprint even started, I had been running tests to see how many people I could recruit with a poll on our signup form at WordPress.com. Fortunately my tests worked out well so I knew I could count on running another one to collect participants for our test. I created a new poll using HotJar and added a little prompt at the end of the poll that asked participants to sign up for other customer research initiatives. This is what it looked like:

Screening participants

Since we wanted to talk to a specific group of people, the link at the end of the poll lead to a Google form which asked visitors to complete a series of questions. We added questions that would help us qualify the people signing up for this test and other tests in the future. Since this was a remote prototype test, one of the key questions was if people were comfortable talking over a video conference.

View all the questions

Sending out invitations

I wanted the email to look like it was coming from a real person so I sent it with Inbox using my full name and company email address a week before we have our test date scheduled. It worked because some people replied back which was great — I was able to do some additional research for another project. I also made sure to add the qualified email addresses to the bcc field so that the recipients couldn’t see any other people being emailed.

These were the key points I included in the email:

• Introduce myself and tell people where I got their email
• Explain what we’re testing along with all the logistics
• Identified what the recipient gets for participating — we decided to offer a $50 Amazon gift card because we felt like we were asking a lot for someone to take an hour out of their busy schedules
• How to sign up for the test.

Coordinating schedules

Getting people across multiple time zones to agree on a date and time over email can be confusing and time consuming. There’s lots of back and forth involved and it’s easy to make mistakes.

That’s why we use Calendly to manage our calendar availability and bookings. It’s easy to setup, offers a number of customizable options, and displays times relative to the timezone of the viewer . You can also automate and personalize event reminders so that you have one less thing to worry about.

Sending a reminder

People’s lives are busy so it can be understandable if someone forgets something they committed to a week ago. Sending a reminder a couple hours before the test is a good way to avoid that and to add any last bits of logistical information required for the test. In my case, I made sure to include:

• the time,
• a link to download the software used for video conferencing — we use Zoom.us for it’s video quality, number of participants, screen sharing, and recording capabilities, and
• a link to join your video conference.

Here’s what my email looked like:

Following up after the test

Lastly, we ended things off with one last email thanking our participants for their time. We included instructions for them to claim their $50 Amazon gift card and also took the opportunity to collect some final feedback about the test so we could make improvements for future participants.

I hope you enjoyed reading this post and remember it next time you’re planning on recruiting participants for your own user test or interview. If you do, let me know how it works out for you by reaching out on Twitter.

#remote-work