RECEIVER-OPERATED CHARACTERISTIC CURVE ANALYSIS OF 2 ALGORITHMS ASSESSING HUMAN GROWTH-HORMONE PULSATILE SECRETION (PULSAR, CLUSTER) - COMPARISON OF PEAK DETECTION EFFICACY

Computer-based peak identification algorithms reduce observer bias in the analysis of pulsatile hormone secretion. With increasing peak detection stringency, an algorithm will detect varying proportions of true-positive and false-positive peaks, determining its receiver-operated characteristics (ROC). To demonstrate that ROC curve analysis can characterize algorithm performance, we analyzed growth hormone (hGH) profiles from 94 children obtained with different hGH assay techniques [radioimmunoassay (RIA), immunoradiometric assay (IRMA)] at different sampling intervals (group A: 1 h/RIA; group B: 1 h/IRMA; group C: 20 min/RIA; group D: 20 min/IRMA), using the PULSAR and CLUSTER algorithms. The area under the ROC curve (AUC) was taken to compare the efficacy of both algorithms over a range of peak recognition stringency thresholds kept constant between algorithms, using hGH noise series for threshold calibration and the results of multiple visual inspection as reference standards. AUC by PULSAR ranged from 0.926 (group C) to 0.961 (group A), indicating good algorithm performance. AUC by CLUSTER ranged from 0.869 (group B) to 0.916 (group D) in the 20-min series, decreasing to 0.756 (group C) and 0.868 (group A) in the 1-hour series. At lower sampling intensity, significant discordant sensitivity existed between algorithms for RIA (p < 0.001) and IRMA (p < 0.0026). When adjusted to a high, assay-specific, comparable stringency, and employed on 20-min sampling hGH data, both the CLUSTER and PULSAR algorithm operated at a similarly high peak detection efficacy. The PULSAR algorithm appears to be more robust when hGH series with lower sampling intensities are analyzed. Until objective validation techniques become generally available, we suggest that different algorithms be tested using reference data sets and ROC curve analysis to select the most efficient algorithm and peak detection stringency threshold for the chosen assay and sampling conditions.