A CASE OF MISTAKEN IDENTITY - SIGNAL AND NOISE IN CONNECTING PERFORMANCE ASSESSMENTS BEFORE AND AFTER GRADUATION FROM MEDICAL-SCHOOL

The authors examine the assumption that there is continuity from one level of training to another in structured and purposeful professional education. Thus, more advanced levels of training are built upon the foundations laid in the preceding levels. While the connection between performance before and performance after graduation from medical school is theoretically rational, such a connection has not been well documented in empirical studies. The issue has been debated but has not been settled because relevant findings are inconsistent. It is argued that these inconsistencies can stem from contaminating factors and the conceptual and methodologic limitations of empirical studies. Such limitations are described in terms of ''noise'' that obscures the maximal value of a true relationship (the ''signal''). Contaminating factors such as the time interval between testings; institutional factors; specialty choices; conceptual dissimilarities between performance measures in medical school and in practice; methodologic limitations such as the shapes of rating distributions, nonlinearity, heteroscedasticity, restriction of range, multicollinearity, voluntary participation, psychometrics of assessment instruments and differing methods of assessments; and a lack of assessments of personal qualities can produce ''noise'' that inhibits the strength of the ''signal.'' While suggesting solutions for extricating some of the tangled web of methodologic and conceptual issues, the authors feel that solutions do not exist for all of the problems. They conclude that researchers should be aware of the limitations if they are to avoid underestimating the ''signal,'' which may fade because of background ''noise.''