Two Simple Post-Test Questions

Sometimes you do things for so long that you forget you do them at all…

This week, I was reminded of a practice we conduct at the conclusion of a usability test. After the participant(s) have finished with their tasks and filled out whatever paperwork we give them to subjectively rate the design, our test facilitator will ask two questions:

1. What are two things about the design that you really liked?

followed by

2. What are two things about the design that you didn’t like?

The first question helps turn the session into a positive. This is really important, especially if things didn’t quite go as smoothly as everyone would like. The participant, along with the team, needs to focus on the positive for a few moments.

The second question helps prioritize. There may have been lots of issues, but what really jumps out? It’s possible these will be the last two problems the participant encountered, but, in my experience, more often then not, they are two issues that really stand out in the participant’s mind. (We often let the participant “tour” the design while attempting to answer these questions, to refresh their memory of what they just experienced.)

In addition to listening to the words they use, we also pay attention to the speed of their answer. If they produce an answer quickly, that tells us one thing. If it takes them a long time to think of a complaint (or a compliment), we give it less weight — it could be they were just fulfilling our request to name two things.

That doesn’t mean, if they take long to arrive at an answer, we disqualify it outright. Often, we’ll just make a note of it and see if it shows up as an issue with anyone else. If we don’t hear it anywhere else, then it probably goes to the bottom of our “issues list” for the design.

We started doing this technique years ago and it’s just become ingrained in our practice. It actually comes from an conflict resolution technique called Stop, Start, Continue. When dealing with individuals who are in constant conflict (such as dysfunctional work relationships), the mediator asks each person to list 2 things they want the other person to stop doing, 2 things they’d like them to start doing, and 2 things to continue doing. In the conflict case, you end on the positive note (continuing good behaviors). When we adapted it to testing, we inverted the polarity so the positive was at the beginning.

This simple technique of asking two questions often provides us some nice insights into where the participant’s mind is at and the lasting impressions they had from the test experience.