This morning i was looking online for articles that could lead me to a method that can be used to measure/quantify usability & user experience. The method i was interested in finding more information about, is about whether it is possible to have a set of good practices that can help in grading an overall system, both in it's prototype stage and even when creating mockups.

Obviously heuristic evaluation and user testing are key elements in developing a good user experience, but i was wondering if before these stages begin and testing is conducted there might possibly be a method for grading your mockups/prototypes internally before going forward to stakeholders and begin testing?

All answers and feedback are welcome as of yet i have been unable to find something that is similar to what i'm describing, i've read up about:

SUM (Single Usability Metric) - http://www.stefanwobben.com/usability/the-single-usability-metric/

System Usability Scale - http://www.measuringusability.com/blog/task-comp-sus.php

  • Sure, you can grade your mockups yourself or the team, but that doesn't mean you match the expectations a common user might have. It's simply an internal perspective, nothing more. But even internal views might be worthwhile - think business objectives.
    – FrankL
    Jan 14 '14 at 18:57
  • What is your goal in measuring/grading/etc? You mention doing this before going to stakeholders and beginning testing, which makes it sound like you are only interested in "grading" until that point and not interested in doing it afterwards.
    – nadyne
    Jan 14 '14 at 19:02

This sounds like a good opportunity to apply hallway usability testing as proposed by Joel Spolsky in his 12 Steps To Better Code article (see popint 12): http://www.joelonsoftware.com/articles/fog0000000043.html

Hallway usability testing is grabbing the next few people near to you, showing them a prototype and getting their feedback. It can be applied to any sort of prototype from a low fidelty wireframe to a full working application. You can present them with any part of a system and a task to complete, whichever part needs some feedback at that point.

It's quick, easy to repeat, doesn't require large resources and can pinpoint many problems with any proposed design. It won't provide you with a score or anything like that but it will give you immediate user feedback which could be converted into a score, i.e. you could grade the success of each person in completing the task set in the test.

For example, you might present a lo-fi wireframe and ask someone 'find the login button' then mesaure the number of seconds it takes them to put their finger on it. At the other end of the scale you might present them with an interface and tell them 'create a new blog post', measuring the points at which they get stuck.

You don't need to use just members from your own team (in fact it's probably best not to). Grab someone from a different department who's making coffee or see if anyone in the communal kitchen has a spare five minutes.

Joel Spolsky references this article which goes into more detail as to why these tests can be very effective:



Your question is very interesting, but in my opinion mockups and prototypes are a means to iterate cheaply, so I don't think you should spend time trying find a universal way to grade your mockups and sort them by grade. Sure this would be great, since we could made 10 prototypes, apply that measure, sort the prototypes and choose the top ones.

The problem however is much more complex, so I think you should only spend time prototyping to get answers to some hypothesis you have formulated.

As an example imagine I am doing an ATM interface. I'll have some hypotheses like " We should only display 7 or less options at the main screen", so I do several (paper or not) prototypes and test them with real users to check if I am totally wrong and if more than 4 options are displayed, users become confused.

The key here is that I'll have perhaps hundreds of hypotheses that I need to validate, so the time I spend making each prototype should be minimal (and therefore my advise is to go with paper). Nowadays with prototyping tools like balsamiq, I find myself spending more time than I should aligning things, finding the correct colors, etc just because the prototyping tools allow it, when I should be spending time testing the prototypes.

In your question you mention heuristic evaluation. I think those methods are great when applied to already existing products, so that you can continuously find usability problems and fix them. You can apply those methods to your prototypes, but might not make much sense.

Edited on 24/02/2014 Bill Buxton et al published a paper, on which they argue that sometimes its preferable to postpone the evaluation of sketches or prototypes, since it can lead to new ideas being created, that otherwise would be disregarded.

Your Answer

