On the measurement of ideation quality

Ideation is a key step in organizational problem solving, so researchers have developed a variety of technological interventions for improving ideation quality, which we define as the degree to which an ideation activity produces ideas that are helpful in attaining a goal. In this paper, we examine the four measures typically used to evaluate ideation quality, including idea-count, sum-of-quality, average-quality, and good-idea-count, and discuss their validity and potential biases. An experimental study comparing three levels of social comparison was used to illustrate the differences in the ideation quality measures and revealed that research conclusions were dependent on the measure used. Based on our analysis of the measures and experimental results, we recommend that only good-idea-count be used as a measure to evaluate ideation treatments and call into question research that has based its findings on the other measures. Finally, we discuss implications for research and other potential approaches to evaluating ideation quality.