SENSITIVITY ANALYSIS OF MODEL OUTPUT - AN INVESTIGATION OF NEW TECHNIQUES

Sensitivity Analysis (SA) of model output investigates the relationship between the predictions of a model, possibly implemented in a computer program, and its input parameters. Such an analysis is relevant for a number of practices, including quality assurance of models and codes, and the identification of crucial regions in the parameter space. This note compares established techniques with variants, such as a modified version of the Hora and Iman importance measure (SANDIA Laboratory Report SAND85-2839, 1989), or new methods, such as the iterated fractional factorial design (Andres, Hajas, Report in prep. for AECL, Pinawa, Canada, 1991). Comparison is made on the basis of method reproducibility and of method accuracy. The former is a measure of how well SA predictions are replicated when repeating the analysis on different samples taken from the same input parameters space. The latter deals with the physical correctness of the SA results. The present article is a sequel to an earlier study in this journal (Saltelli, Homma, Comp. Stat. and Data AnaL 13 (1) 1992, 73 94 of limitations in existing SA techniques, where the inadequacy of existing schemes to deal with non-monotonic relationships within the model was pointed out. International benchmark test models were taken from the field of performance analysis of nuclear waste disposal in geological formations. The results based on these models show that the modified version of the Hora and Iman method proposed in this paper is extremely robust, when compared with the other existing statistics, even in the presence of model non-monotonicities. This importance measure is accurate, although its application is demanding - in terms of computer time - for system with very large numbers of input parameters. Its predictions are also among the most reproducible. The newly proposed iterated fractional factorial design appears to score the best in reproducibility. The accuracy of this latter method demands further investigation