Measuring balance and model selection in propensity score methods

Purpose Propensity score (PS) methods focus on balancing confounders between groups to estimate an unbiased treatment or exposure effect. However, there is lack of attention in actually measuring, reporting and using the information on balance, for instance for model selection. We propose to use a measure for balance in PS methods and describe several of such measures: the overlapping coefficient, the Kolmogorov-Smirnov distance, and the Levy distance. Methods We performed simulation studies to estimate the association between these three and several mean based measures for balance and bias (i.e., discrepancy between the true and the estimated treatment effect). Results For large sample sizes (n = 2000) the average Pearson's correlation coefficients between bias and Kolmogorov-Smirnov distance (r = 0.89), the Levy distance (r = 0.89) and the absolute standardized mean difference (r = 0.90) were similar, whereas this was lower for the overlapping coefficient (r = -0.42). When sample size decreased to 400, mean based measures of balance had stronger correlations with bias. Models including all confounding variables, their squares and interaction terms resulted in smaller bias than models that included only main terms for confounding variables. Conclusions We conclude that measures for balance are useful for reporting the amount of balance reached in propensity score analysis and can be helpful in selecting the final PS model. Copyright (C) 2011 John Wiley & Sons, Ltd.