Expressing preferences in a principal-agent task: A comparison of choice, rating, and matching

One of the more disturbing yet important findings in the social sciences is the observation that alternative tasks result in different expressed preferences among choice alternatives. We examine this problem not from the perspective of an individual making personal decisions, but from the perspective of an agent trying to follow the known values of a principal. In two studies, we train people to evaluate outcomes described by specific attributes and then examine their ability to express these known values with three common tasks: ratings of individual alternatives, choices among triples of alternatives, and matching pairs of alternatives to indifference. We find that each preference assessment method has distinct strengths and weaknesses. Ratings are quick, robust at following known values, and are perceived as an easy task by respondents. However, because ratings require projection to an imprecise response scale, respondents have difficulty when applying them to more complex preference structures. Further, they place too much weight on negative information, a result that is consistent with reference-dependent loss aversion. Choice is perceived as the most realistic task and the one about which people feel the most confident. However, choices exhibit the most negativity, which, in addition to flowing from the same perceptual bias of ratings, may be exacerbated by a screening strategy that excludes alternatives possessing the lowest level of an attribute. Finally, the matching task takes the most time and is perceived to be the most difficult. It shows minimal biases, except for one glaring flaw, a substantial overweighting of the matching variable. This bias is consistent with a well-known compatibility bias and suggests that agents can learn to use a matching task appropriately for all attributes except the matching variable itself. The article concludes with a discussion of the theoretical mechanisms by which these biases infiltrate different elicitation modes and a summary of managerial implications of these results. (C) 2001 Elsevier Science.