Reliability of a hospital utilization review method in Turkey

Objective. To determine whether the Appropriateness Evaluation Protocol (AEP) is reliable in Turkey. Methods. Three reviewers, two physicians and one nurse each reviewed 196 patient-days concurrently by using the AEP at three hospitals, two of which were teaching hospitals. Inter-reviewer reliability was assessed both for all cases reviewed (overall agreement), and for only those judged inappropriate by at least one reviewer (specific agreement). In addition, overall agreement between pairs of reviewers was evaluated by the Kappa statistic. Results. The overall agreement between pairs of reviewers was very high: 93.4-95.9%, and it was similar between all pairs. The level of overall agreement was highly statistically significant: k = 0.725-0.833, P<0.001. The specific agreement rates ranged from a low of 61.8% to a high of 75%. Conclusions. These results show, for the first time, that the AEP method is reliable in Turkey.