The many faces of replication

Replication is one of the three cornerstones of inference from experimental studies, the other two being control and randomization. In fact, replication is essential for the benefits of randomization to apply. In addition to ordinary replication, the repetition of treatments within a study, two other levels of replication have been identified. Pseudoreplication, a termed coined by Stuart Hurlbert, generally involves making multiple measurements on experiment units (which is commendable) and treating them as if they reflected independent responses to treatment (which is erroneous). Metareplication is a higher level of replication in which entire studies are repeated. Scientists are too much concerned about analysis of data within studies and too little concerned about the repeatability of findings from studies conducted under a variety of conditions. Findings that are consistent among studies performed at different locations at different times with different investigators using different methods are likely to be robust and reliable.