Assessing surrogacy of data sources for institutional comparisons

Can administrative claims data, Z, serve as a surrogate for better clinical data, X, when assessing institutional performance? We consider an analysis of I hospitals, each of which involves an adjusted outcome. In the ith hospital, we denote the true association between the outcome and the risk factors using one data source by θi(X), the true associatio n between the outcome and the risk factors using the other data source by γi(Z), and assume we have estimates of each available. Within hospital i, the estimated association parameters are jointly normally distributed such that conditional on γi(Z), a simple linear rel ationship exists between θi(X) and γi (Z). Methods are illustrated using mortality rates for 181,032 elderly US heart attack patients treated at 4322 hospitals. We find a strong linear relationship between the hospital standardized mortality rates adjusted by risk factors found in administrative claims data and rates adjusted by risk factors found in medical charts (posterior mean [95% interval] for slope: 0.997 [0.965,1.028]). However, the absolute and relative differences between the two sets of rates increase as hospital volume increases. For typically-sized standard deviations of claims-based rates, there is reasonable certainty of quality problems when the hospital's claims-based rate is 0.72 times or smaller than the national mean or 1.45 times or greater than the national mean. Fewer hospitals are classified as either low-mortality or high-mortality hospitals when using claims-based estimates compared to chart-based estimates. © Springer Science+Business Media, LLC 2007.