Meta-analysis and aggregation of multiple published prediction models

被引:49
作者
Debray, Thomas P. A. [1 ]
Koffijberg, Hendrik [1 ]
Nieboer, Daan [2 ]
Vergouwe, Yvonne [2 ]
Steyerberg, Ewout W. [2 ]
Moons, Karel G. M. [1 ]
机构
[1] Univ Med Ctr Utrecht, Julius Ctr Hlth Sci & Primary Care, NL-3508 GA Utrecht, Netherlands
[2] Erasmus MC, Ctr Med Decis Sci, Rotterdam, Netherlands
关键词
prediction research; risk prediction models; aggregation; updating; validation; logistic regression; multivariable; external validation; LOGISTIC-REGRESSION ANALYSIS; INDIVIDUAL PARTICIPANT DATA; DEEP VENOUS THROMBOSIS; EXTERNAL VALIDATION; PROGNOSTIC MODELS; CARDIOVASCULAR-DISEASE; VEIN THROMBOSIS; PRIMARY-CARE; RISK; IMPACT;
D O I
10.1002/sim.6080
中图分类号
Q [生物科学];
学科分类号
07 ; 0710 ; 09 ;
摘要
Published clinical prediction models are often ignored during the development of novel prediction models despite similarities in populations and intended usage. The plethora of prediction models that arise from this practice may still perform poorly when applied in other populations. Incorporating prior evidence might improve the accuracy of prediction models and make them potentially better generalizable. Unfortunately, aggregation of prediction models is not straightforward, and methods to combine differently specified models are currently lacking. We propose two approaches for aggregating previously published prediction models when a validation dataset is available: model averaging and stacked regressions. These approaches yield user-friendly stand-alone models that are adjusted for the new validation data. Both approaches rely on weighting to account for model performance and between-study heterogeneity but adopt a different rationale (averaging versus combination) to combine the models. We illustrate their implementation in a clinical example and compare them with established methods for prediction modeling in a series of simulation studies. Results from the clinical datasets and simulation studies demonstrate that aggregation yields prediction models with better discrimination and calibration in a vast majority of scenarios, and results in equivalent performance (compared to developing a novel model from scratch) when validation datasets are relatively large. In conclusion, model aggregation is a promising strategy when several prediction models are available from the literature and a validation dataset is at hand. The aggregation methods do not require existing models to have similar predictors and can be applied when relatively few data are at hand. Copyright (c) 2014 John Wiley & Sons, Ltd.
引用
收藏
页码:2341 / 2362
页数:22
相关论文
共 85 条
[61]  
Schmid C.H., 2005, Encyclopedia of biostatistics, DOI [10.1002/0470011815.b2a13049, DOI 10.1002/0470011815.B2A13049]
[62]   An Asian Validation of the TIMI Risk Score for ST-Segment Elevation Myocardial Infarction [J].
Selvarajah, Sharmini ;
Fong, Alan Yean Yip ;
Selvaraj, Gunavathy ;
Haniff, Jamaiyah ;
Uiterwaal, Cuno S. P. M. ;
Bots, Michiel L. .
PLOS ONE, 2012, 7 (07)
[63]  
Steyerberg E, 2009, Clinical prediction models: a practical approach to development, validation, and updating, DOI 10.1007/978-0-387-77244-8
[64]   Validation and updating of predictive logistic regression models: a study on sample size and shrinkage [J].
Steyerberg, EW ;
Borsboom, GJJM ;
van Houwelingen, HC ;
Eijkemans, MJC ;
Habbema, JDF .
STATISTICS IN MEDICINE, 2004, 23 (16) :2567-2586
[65]   Prognostic modeling with logistic regression analysis: In search of a sensible strategy in small data sets [J].
Steyerberg, EW ;
Eijkemans, MJC ;
Harrell, FE ;
Habbema, JDF .
MEDICAL DECISION MAKING, 2001, 21 (01) :45-56
[66]  
Steyerberg EW, 2000, STAT MED, V19, P141, DOI 10.1002/(SICI)1097-0258(20000130)19:2<141::AID-SIM334>3.0.CO
[67]  
2-O
[68]   Stepwise selection in small data sets: A simulation study of bias in logistic regression analysis [J].
Steyerberg, EW ;
Eijkemans, MJC ;
Habbema, JDF .
JOURNAL OF CLINICAL EPIDEMIOLOGY, 1999, 52 (10) :935-942
[69]  
Steyerberg EW, 2000, STAT MED, V19, P1059, DOI 10.1002/(SICI)1097-0258(20000430)19:8<1059::AID-SIM412>3.3.CO
[70]  
2-S