Are road safety evaluation studies published in peer reviewed journals more valid than similar studies not published in peer reviewed journals?

The peer review system of scientific journals is commonly assumed to prevent seriously flawed research from getting published. This paper compares the quality of 44 road safety evaluation studies published in peer reviewed journals to the quality of 79 evalutation studies dealing with the same safety measures, but not published in peer reviewed journals, in terms of seven criteria of study validity. Studies were scored for validity in terms of (1) sampling technique, (2) total sample size, (3) mean sample size for each result, (4) specification of accident or injury severity, (5) study design, (6) number of confounding factors controlled and (7) number of moderator variables specified. Confounding factors are all factors that distrurb the attribution of a causal relationship between the safety measure being evalutated and the observed changes in safety, moderator variables are all variables that influence the size of the effect of the safety measure. Very few statistically reliable differences in study validity were found between studies published in peer reviewed journals and studies not published in such journals. There was, at best, a weak tendency for studies published in peer reviewed journals to score higher for validity. An interaction was found between author affiliation and type of publication with respect to study validity. Studies published in peer reviewed journals by authors who were at a university scored highest for validity. For a number of reasons, this study must be regarded as exploratory and its results as indicative only. The study does, however, point to a line of research that might be worth pursuing in larger and more rigorous studies. (C) 1998 Elsevier Science Ltd. All rights reserved.