Prioritizing usability problems: data-driven and judgement-driven severity estimates

Software-ergonomic system analysis often reveals numerous usability problems. Given that system design suffers from limited resources, the prioritization of usability problems seems inevitable. Surprisingly enough, prioritization is not in the focus of scientific interest. Within this paper, approaches to prioritization relying on severity estimates will be presented. Two of the approaches, namely priorities based on data about the impact of a problem? (data-driven) and priorities based on judgements of interest group members (judgement-driven) will be further explored. In the data-driven approach total problem-handling time caused by a usability problem is presented as a measure of severity. The major disadvantage of the data-driven approach is its costs. A possible alternative are severity estimates based on judgements by members of involved interest groups. The first of two studies shows how to obtain judgement-driven severity estimates and reveals a fundamental lack of correspondence between data-driven and judgement-driven severity estimates. The second study supports the notion that the lack of correspondence may stem from a difference between assumptions of the data-driven approach and the naive judgement model of interest group members in the judgement-driven approach. A hypothetical model for severity estimates by interest group members is presented.