ASSESSING SOFTWARE DESIGNS USING CAPTURE-RECAPTURE METHODS

Sometimes complex software systems fail because of faults introduced in the design stage of the development process. Design reviews can remove some of these faults but often a few remain undiscovered until the software is developed further. The number of faults not discovered by the design review can be estimated by using capture-recapture methods. Since these methods were developed for wildlife population estimation, the assumptions used to derive them do not match design review applications. We report on a Monte Carlo simulation to study the effects of broken assumptions on maximum likelihood estimators (MLE's) and jackknife estimators (JE's) of faults remaining. We find that the MLE performs satisfactorily if faults are classified into a small number of homogeneous groups. Without grouping, the MLE can perform poorly but it generally does better than the JE.