Characterizing Structural Uncertainty in Decision Analytic Models: A Review and Application of Methods

Background: The characterization of uncertainty is critical in cost-effectiveness analysis, particularly when considering whether additional evidence is needed. In addition to parameter and methodological uncertainty, there are other sources of uncertainty which include simplifications and scientific judgments that have to be made when constructing and interpreting a model of any sort. These have been classified in a number of different ways but can be referred to collectively as structural uncertainties. Materials and Methods: Separate reviews were undertaken to identify what forms these other sources of uncertainty take and what other forms of potential methods to explicitly characterize these types of uncertainties in decision analytic models. These methods were demonstrated through application to four decision models each representing one of the four types of uncertainty. Results: These sources of uncertainty fall into four general themes: 1) inclusion of relevant comparators; 2) inclusion of relevant events; 3) alternative statistical estimation methods; and 4) clinical uncertainty. Two methods to explicitly characterize such uncertainties were identified: model selection and model averaging. In addition, an alternative approach, adding uncertain parameters to represent the source of uncertainty was also considered. The applications demonstrate that cost-effectiveness may be sensitive to these uncertainties and the methods used to characterize them. The value of research was particularly sensitive to these uncertainties and the methods used to characterize it. It is therefore important, for decision-making purposes, to incorporate such uncertainties into the modeling process. Conclusion: Only parameterizing the uncertainty directly in the model can inform the decision to conduct further research to resolve this source of uncertainty.