How to Conduct Usability Testing in Six Steps

Design testing will help you build outstanding products that work for people.

User testing is known by many names: product testing, design testing, usability testing, design validation, etc. It’s about the critically important process of testing your designs with real users in real scenarios. Once you deeply understand user concerns and usability issues, you become better at correcting them.

User testing—which is part of a user-centered design philosophy—is one of the most important aspects of a designer’s job. It’s not just about stunning visuals (the surface). It’s about solving a problem effectively; something that can only be accomplished by repeated testing and with many iterations on the design. It’s an excellent way to gain valuable insights into what works and what doesn’t.

I want to make sure the design works well, and that all assumptions and hypotheses are validated—testing with users has become one of the most important steps of my design process.

User testing is like watching over someone’s shoulder while they interact with your app or site, except that you get to ask questions along the way and nudge them in the direction you want to explore. It’s not that expensive or hard to do. Unfortunately, not enough design/product teams do it.

We’ll go through the reasons for—and the six essential stages of—usability testing.

What Is Usability Testing?

There’s an old adage in the UX community: “You are not the user.”

I’ll never forget my first user testing experience. For two months, I worked hard fine-tuning a prototype for a web application. I wanted to make it perfect and I was naively, unabashedly sure it would work. I could not have been more wrong. Like so many other designers, I had become infatuated with my own creation.

The first person I tested became confused after the second step—and in one stroke, my illusions were destroyed. I was devastated.

That day, I went through five website usability testing sessions. At the end of it all, I was forced to accept the fact that perhaps I wasn’t as good a designer as I thought I was.

The next day, I replaced my polished prototype with quick interactive wireframes that represented a new and improved design concept based on insights I gained from the previous day’s user testing. The results were impressive—even a novice user was able to use a fairly complex application.

That first testing experience had such an impact on me as a designer, I now test every single design. It’s always been worth it, and I’m always surprised by the things I got wrong.

It’s decidedly better to make mistakes on a quick prototype than on the finished product. Sadly, many companies invest huge amounts of money into untested hypotheses and applications.

Wrong design decisions become very apparent on a finished product—unnecessary mistakes that need to be fixed at a huge cost. The final cost of deploying untested applications is enormous.

The Six Basic Elements of User Testing

To go through the proper user testing process you need to do the following:

• Create a prototype
• Come up with a test plan
• Recruit people (target users)
• Find a suitable location
• Moderate the test
• Document the test result

Creating a Prototype

A prototype is an early model of a product—usually with limited functionality—built to test a design concept; a thing to be replicated or learned from. There are multiple ways to create prototypes, and there are many tools, including offline software and online apps.

Different fidelities and approaches are appropriate for different situations.

There is the very basic way—draw some sketches on a piece of paper, take pictures of them and then join them together using a tool like Adobe Experience Design. You can do this for both desktop and mobile apps.

Alternatively, you can create interactive wireframes—stringing wireframes together in an online prototyping tool like InVision.

I create multiple variations and build several completely different prototypes for the same project. I decide which one is best after I put them together. More often than not, once I start working with them, I discover solutions that I didn’t think of when I created the first prototype.

My initial prototyping tool was Axure RP. It’s very complex and takes about six months to master. However, you can learn the basics in a few days and create simple interactive prototypes.

I have also tried some solutions such as InVision or UXpin. They are web-based, innovative, and much less expensive. They’re not as feature-laden and flexible as Axure, but they’re much simpler to use. You can also share your prototype easily with clients and team members.

Recently, Marvel has been my tool of choice. It’s a great mobile and web application, it’s incredibly easy to create and share prototypes, and it also has a handy Sketch plugin.

There is a wide variety of prototyping tools out there. Here’s a great comparison site: http://www.prototypr.io/prototyping-tools.

Putting Together a Product Usability Testing Plan

No matter what kind of digital product you’re designing, it’s important to lay out a plan for the testing; otherwise, you will end up wasting a lot of time. At a basic level, the plan should outline:

• What is being tested (features, functionalities, tasks)
• How it is going to be measured (the success or fail-rate of the test in specific areas)

Let’s say I’m testing a mobile app for a public transportation route planner. I designed three main features of the app: journey planning, ticket ordering, and seat reservations.

I have a group of target users in mind who usually travel everywhere by bus. During the software usability testing session, I’ll go into specifics and try to find out from my users if certain features and functionalities are user friendly—i.e., “apparent” (obvious to the user) and easy to use.

Writing up the test plan, I go through the prototype and jot down some questions. For example:

• Are users able to plan their journey effectively and quickly?
• Can they order tickets easily?
• Are they able to reserve a seat?

Let’s say that fifteen people agreed to join the testing. Before the session begins, I create a spreadsheet with five columns:

• Column 1: The names of the participants,
• Columns 2-4: three features of the app to be tested,
• Column 5: if a participant is struggling with using a given feature, I take note of it and I write down the reason for the confusion.

Working with this spreadsheet, I can evaluate the success of my prototype. If fifteen people test three features, that makes for 45 (15x3) tasks. If people accomplish 38 tasks, I have a success rate of 85%.

Recruiting People, a.k.a. “Users”

If you’re working on a project with existing customers, recruiting should be easy. Reach out to the sales team or the stakeholders and get some customer contacts.

For a small personal project or an in-house company project, try reaching out to your friends on Facebook who fall into the target audience. You could also ask for a referral to their contacts. Other members of the team can do the same. It’s important to find unbiased participants who will not just want to be “nice to you,” i.e., like everything they see. You need unbiased feedback—good or bad—to really get value out of the test.

If you’re working with an external client on a bigger project, I recommend using services such as usertesting.com, validately.com, clicktale.com, or verifyapp.com.

Sometimes it can be difficult to explain what you’re looking for when recruiting potential users. Participating in user testing isn’t familiar to most people. My request usually goes like this:

“Hey Walt! I know you’re really busy, but I’m hoping you could spare me half an hour and help me out with a project I’m working on. It’s a public transportation journey planning app. I know you frequently travel by bus, and I’d like to see how you would use it.”

It’s best to have a minimum of 3-5 tests (remote or in-person). That should get you enough material for the first iteration. If you can muster it, you can offer participants coffee/tea and a snack, a gift card, or a discount code from the company in return for their participation.

It’s a good idea to schedule the tests with a 30-45 min break between them. You might find that you approached your design from a bad angle and you can fix the biggest problems on the spot before the next tester arrives. Testing is intense, and you will also need a breather to collect your findings, make notes, and relax before the next round.

Finding a Suitable Testing Location

I recommend testing in person because you can read body language and subtle signals like tension and sighs, or catch things such as people making faces because they are struggling. This is an opportunity to step in and ask if they’re confused about something.

Additionally, people tend to be more focused during in-person settings. If you’re doing remote testing over Skype or Hangout, they can become distracted and things might slip by you.

Finding a compatible meeting place with the least distraction or opportunity for interruption is crucial. You need to find a setting where people can stay focused on the test.

I’d prefer not to bring participants to my client’s offices—it’s conceivable the corporate environment could make a them feel confined and as if they are under observation. Neither is visiting people in their homes an optimal situation. A pet, a child, or family member may distract your test users.

In my opinion, a cafe is the perfect environment. There’s music playing, it smells great, and the coffee keeps everyone alert. Away from personal commitments and distractions, test subjects tend to relax—also, it often helps them with the testing process to feel as if they are in a real world scenario.

You can even do testing outside! Sometimes you may want to put your potential users in a real, contextually accurate setting—like riding a bus together while looking at your bus journey planning prototype.

Running the Test

The quality of testing is heavily influenced by your ability to run the test well and communicate effectively (called “moderating the test”). This is also where preparation and your test plan comes in. If you haven’t tested anything before, I recommend a few no-pressure run-throughs with your partners or family members.

Practice asking the type of questions you’re going to ask, and you’ll be able run your test more effectively.

It would be a mistake to test everything in a warm, cozy environment with unlimited time to accomplish tasks. Give your users context and remind them to imagine they are using your app in a real-life situation—while exercising, commuting, or shopping, for example.

It’s also important to thank your participants for taking the time to help you. Remind them that the reason you’re doing these user feedback sessions is because you are eager to optimize your site/app/platform’s usability for the best user experience.

Make sure you tell them that they’re not the ones being tested. It’s the design.

If you’re doing a remote test, record it with screen recording software. If in person, use a video camera or an audio recorder. You’ll be amazed how much you missed!

By recording the sessions, you’ll find you are more relaxed because you don’t have to take notes. You are more able to concentrate on establishing good rapport and making the test comfortable for your participants. You can document everything later from the recordings.

Make sure to tell your test subjects that recordings are confidential, for internal purposes only and that it is you alone who will watch/listen to them.

MacOS users can use QuickTime for screen capturing. I use Screenflick ($29). It can record the screen and the webcam at the same time, trim the recording, and export the results really well. There are also a few apps available for iOS.

In order to gain real insight into your users’ needs, behaviors, goals, and frustrations, ask open-ended questions that do not lead them. Here are some examples:

• I noticed a bit of hesitation there, what stopped you?
• What do you think this button is going to do?
• What’s most appealing about this product?
• What’s the hardest part about using this product?
• Was there anything surprising or unexpected about this product?
• What could be done to improve this product?
• What may be missing? What else would you like to see?
• What do you like/dislike about the way it works?
• How do you think this product is going to help you?
• Would you use this product today?
• Why do you think someone would use this product?

Documenting Your Test Results

Distill and summarize your findings in a short, one-page report from the test documentation spreadsheet I mentioned earlier. This will help you focus on the issues that caused people most trouble. The areas needing improvement will become apparent very quickly. Typically, my test artifacts consist of:

• The spreadsheet with notes
• An audio recording
• A screen recording
• A recording of the test participant from a webcam

When your test is documented properly, you can study it further and have it guide your next iteration. You can also use the test results with stakeholders to back up your next design decisions.

Embrace Testing and Enjoy It. It Will Be Fun!

In conclusion, remember two things:

1. The rule of thumb: If it takes more than ninety minutes to build a prototype, you’re trying to do too much.
2. Once the success rate of the test goes over 90%, stop testing. The app/site/platform should now be very usable and “useful.” MVP time!

In the beginning of this article, I said that the results of my first testing devastated me. It shook my trust in my own abilities. I realized then that trial and error is a natural part of the process. It’s not true that a professional doesn’t make mistakes. A professional learns from them quickly, shortens the iteration loop, and perfects the design in the process.

To designers: Enjoy the testing process—it improves your designs! Test out your ideas with real users in real scenarios as quickly as possible. The insights you gain from observing how actual users use your product are priceless. Every single iteration is an important step forward.

To clients: Don’t make designers finish projects without testing. Make the most of opportunities to evaluate your products before the first line of code is written. It might take one extra week and cost a bit more, but you’ll save in the end and gain a hundred times over by doing it!