Early on in my career as a user experience analyst and researcher, I was part of a challenging project that forever changed the way I thought about product, research, and how we involve customers in our process.
One of my clients at the time was a large Vancouver-based agency that specialized in e-commerce solutions. They had just completed a very high profile project: Launching the first online sales portal for one of Hong Kong’s largest mobile phone providers, who was also amongst the first in Asia to sell the newly released and much vaunted iPhone 4. The new site and iPhone launch had garnered a significant amount of media attention and everything had gone exactly as planned. The team celebrated what seemed a perfect victory.
However two weeks post launch, the project was in chaos: Only a handful of phones had been sold via the flashy new site. The client was in panic mode. Highly qualified traffic was flowing to the site. Nothing in QA or within the analytics data indicated anything was broken or technically wrong—we could see customer after customer adding phones and service plans into their cart and then promptly abandoning them during checkout. Out of answers and desperate to get control of the situation, the client put me on a plane to Hong Kong to conduct an audit and in-person usability testing that they hoped might shed some light on the rapidly deteriorating situation.
After two days on the ground, I had sat next to at least a dozen participants as they gamely put phones and plans into their carts, plugged in the dummy credit card I’d provided and completed checkout without a single issue. I had landed thinking that there must be some simple usability issue we’d overlooked, but after two days of product-focused sessions revealing essentially nothing my mood was increasingly bleak. Feeling hopeless and a bit frantic, I noticed Joyce, the young local translator who was my helper for the week, tapping away on her new iPhone. “Joyce,” I asked her, “where did you get your iPhone?”. “In a shop.” she replied. When I inquired why she hadn’t purchased it online, Joyce told me it just wasn’t something she would buy that way. As I dug deeper into her purchase process, I immediately learned some very interesting facts about Hong Kong.
Prior to this engagement, neither myself nor my client had worked with a company fully based in Asia. On top of this, the mobile provider itself had recently been acquired by a large Australian telco–who had promptly re-staffed its leadership ranks with a team of expats also brand new to Hong Kong. It began to dawn on me that none of us had spent much time actually getting acquainted with the local customs or people we had been designing for.
Joyce told me that she had gone to two or three different shops near her home, negotiating for herself the best possible deal on the phone and discounts on the required data plan. She was able to get her phone straight away, with no wait on shipping, but she could also partake in a custom very specific to and important in the retail landscape of Hong Kong: Bargaining and negotiating. While the shiny new e-commerce portal was loaded with all the pertinent information and features required to buy a new iPhone, it missed the mark in terms of its intended market. In a densely packed city where you could throw a rock and hit a dozen competing mobile phone shops or kiosks, customers had little incentive to purchase via an e-commerce portal that might take weeks to deliver their phone–and not provide any ability to negotiate the freebies or bonuses they knew competing retailers would certainly offer them.
While the teams involved had delivered a great project on time and on budget, we hadn’t brought potential customers into the mix until it was far too late. We conducted usability testing of the solution and felt certain delivery and execution had been aced, but we had asked the wrong questions and also made some pretty big assumptions along the way.
“Start from curiosity. Welcome and seek out difference.” – Automattic Design Principle
Generative, foundational research helps us to understand our customers’ context and gives us a clear view of their experiences and struggles. By starting earlier and talking to potential users before we ever start to formulate a solution, we can be more strategic about product design—taking into account customers’ cultures, beliefs, perceptions and actual points of friction before charting any course. We can avoid building the perfect car for a customer who only needs a bicycle. In the case of “The $2M E-commerce Portal That Couldn’t”, the team had executed the perfect solution, but didn’t solve for its intended market. Had we spent time getting to know some locals and the city’s highly idiosyncratic commercial culture, we would have saved ourselves (and the client) a lot of effort, money and panic.
After this retail revelation, I shifted my approach and spent the next day interviewing participants about their purchasing preferences and learning more about the ways they shopped and made decisions about new electronics. As an experiment, we quickly added a printable coupon to the site with a special offer for a deeply discounted data plan and savings off of a second phone—and the next day our coupons began appearing in droves in retail shops and kiosks across the island. We were able to show that the new site actually was driving sales, just not in the way the executive team had envisioned. It wasn’t a complete save of the project, but for me it drove home the fact that early formative research was something that couldn’t be overlooked in the future.
“Search for and tell stories about people, not just data.” – Automattic Design Principle
While it’s important to vet our product concepts and prototypes with customers, when we only evaluate solutions we may be prevented from solving the right problem. At Automattic, we’ve recently been ramping up our efforts in this area of generative research, engaging in illuminating, product-agnostic discussions with both existing and potential customers. We are involving these people earlier in our process, not only learning about the ways they work, but also their day-to-day lives, dreams, hopes and fears. While this research is challenging and may not provide immediate answers to all of our product-specific questions, it absolutely puts us closer to making sure we are at least asking the right ones.
—
Photo is author’s own. Saturday morning in Aberdeen – September 2010.
Automattic Design 