CONSENSUS DIAGNOSIS - A SIMULATION STUDY

The derivation of consensus among a panel of qualified experts is a long-standing and far-reaching problem. One of its many guises is the consensus diagnosis. Consensus diagnoses arise in situations in which several experts, acting in good faith and towards a common goal, contribute their opinions about the relative merits of a series of competing hypothesis. A single decision maker must then combine their responses and make a decision without allowing further discussion among the contributors. The paper simulated consensus diagnoses by allowing an oracle to generate ''opinions'' based on universal background knowledge and all available information about the specific problem being diagnosed. Contributors' opinions were then generated by perturbing the oracle's opinion; the size of the perturbation depended on the contributor's degree of expertise. Several different aggregation functions-the linear opinion pool, the logarithmic opinion pool, the conjugate method, and a formula proposed by Bordley-were then used to reclaim the oracle's opinion from those of the contributors. The performance of these functions was compared as panel size and hypothesis-set size varied from two to ten. Comparative and individual analyses yielded a clear and consistent recommendation to managers assembling panels under circumstances similar to those of this study: Small, simple methods work best.