Usability Testing: Are You Asking The Right Questions?

Usability testing is one of the most effective tools to aid in uncovering problem areas or stuck spots in a website’s design and structure. While any measure of testing is helpful, different types of testing and the way in which questions are asked can greatly affect user response, and ultimately, the outcome of solutions that will be put in place.

The first thing to determine is whether to take a scripted or un-scripted approach to testing. Scripting is recommended for testing specific use cases and offers many benefits including delivering the most controlled environment. Reporting from scripted testing delivers both quantitative and qualitative data, all dependent on how the script is crafted.

Setting the Scene

Foremost, its important to indicate what areas of focus will be tested – this could be anything from the name of a website and how it will register with users to how the product/website functions. Here is where specifics are key: how much will be covered in the test, ensuring that there are no surprises moving forward.

Determine whether the test will be moderated or un-moderated. Moderated tests are ideal for complex sites such as ecommerce that may be content or pathway heavy – however, moderators will be required and should be chosen carefully. An unmoderated test can lend a more ‘realistic’ approach to where and how the user will truly behave within the site.


Developing A Structure

Define your test goals by determining concerns, questions, needs and problems that will be driving the test. The questions asked in preparation for the test are just as important as the actual questions asked when testing. These can range from the widespread, general topic or drilled-down, detailed and specific concerns. Consider internal questions such as:

  • Can users access all of the relevant elements and tabs on our home page?
  • Will our users find a specific tab or button on that same page or will this button have a better effect on the other side of the page?
  • Which call-to-action buttons are users clicking on?
  • How can we optimize conversion when products are already in the shopping cart (i.e. abandoned carts)?

As queries for the test are developed, separate complicated or multi-step questions to give the user ‘room to breathe’. If multiple complex problems are asked one after another, users may become fatigued and either skip questions or leave answers that can negatively affect the outcome of the test.

General questions are not recommended as vague answers will be returned. Ask a series of very specific questions about one subject which will garner a more accurate depiction of the users feeling. Using either a scaled answer (Would use again, Would never use again), (Very Satisfied, Not Satisfied).


Testing should be done at least once a month, so questions can be a work in progress. Define the main goals or problems to achieve or fix and start there. By conducting ongoing testing, a site can gather fresh information and continue to optimize on a regular basis. This will create a healthy and incremental approach resulting in the probability of increased user experience and heightened conversion.

Vanessa Petersen

As Executive Director of Brand Strategy based in ArtVersion’s Portland office, Vanessa manages both national and West Coast accounts. She brings with her a passion for strategy with extensive experience in creative marketing, brand growth and development. She is a true advocate of storytelling and brand building from the inside out, and her keen knowledge of the marketplace, trend patterns and usability lead her to identify opportunities that can inspire powerful design strategies.

More Posts - Website - Twitter - Facebook - LinkedIn


Leave a Comment