Would You Bet Your Life Savings On It?
The point of a usability test is to determine areas in the design we’d like to improve. Often, when the testing is complete, the team expects us to put together a list of recommendations for them to act on. Each recommendation should point to an improvement which will benefit the design and therefore our organization.
Improvements happen when the design truly eliminates frustration. Organizations benefit from those improvements when the eliminated frustration increases sales (due to a better user experience) or reduces expenses (due to decreased support or development costs).
One problem we often see, however, is when the user researchers get overzealous in their improvement suggestions. They start to make recommendations in cases where there may not be any clear evidence the change will eliminate frustration or improve the design in a measurable way.
The worst case scenario is when someone issues a poorly thought out recommendation, which turns out to make the design more frustrating. When this happens, it hurts the reputation of the user research effort and puts into question other recommendations.
At the recent CHI conference, our good friend Meghan Ede, who runs the user research effort at Symantec, told us she’s instructed her team of researchers to ask the following of every recommendation they write: “Would you bet your life savings on this recommendation improving the design?” They remove any recommendation which doesn’t meet this criteria from the final presentation.
Meghan reports, since she’s instituted the policy of asking this question for every recommendation, the number of recommendations has dramatically decreased, but the quality of their results have substantially improved. The researchers are now more confident when they report their results and the teams are less argumentative.
What do you think of Meghan’s approach?