What is the probability of replicating a statistically significant effect?

被引:55
作者
Miller, Jeff [1 ]
机构
[1] Univ Otago, Dept Psychol, Dunedin, New Zealand
关键词
CONFIDENCE-INTERVALS; PSYCHOLOGICAL-RESEARCH; GAMBLERS FALLACY; HOT HAND; P VALUES; POWER; INFERENCE;
D O I
10.3758/PBR.16.4.617
中图分类号
B841 [心理学研究方法];
学科分类号
040201 ;
摘要
If an initial experiment produces a statistically significant effect, what is the probability that this effect will be replicated in a follow-up experiment? I argue that this seemingly fundamental question can be interpreted in two very different ways and that its answer is, in practice, virtually unknowable under either interpretation. Although the data from an initial experiment can be used to estimate one type of replication probability, this estimate will rarely be precise enough to be of any use. The other type of replication probability is also unknowable, because it depends on unknown aspects of the research context. Thus, although it would be nice to know the probability of replicating a significant effect, researchers must accept the fact that they generally cannot determine this information, whichever type of replication probability they seek.
引用
收藏
页码:617 / 640
页数:24
相关论文
共 70 条
[1]   On the surprising longevity of flogged horses: Why there is a case for the significance test [J].
Abelson, RP .
PSYCHOLOGICAL SCIENCE, 1997, 8 (01) :12-15
[2]  
Agresti A., 2002, CATEGORICAL DATA ANA, DOI [10.1002/0471249688, DOI 10.1002/0471249688]
[3]  
[Anonymous], UNDERSTANDING STAT, DOI DOI 10.1207/S15328031US0304_9
[4]  
[Anonymous], 1986, EPIDEMIOLOGY RESOUR
[5]   The hot hand fallacy and the gambler's fallacy: Two faces of subjective randomness? [J].
Ayton, P ;
Fischer, I .
MEMORY & COGNITION, 2004, 32 (08) :1369-1378
[6]  
Batanero C., 2000, MATH THINK LEARN, V2, P75, DOI DOI 10.1207/S15327833MTL0202_4
[7]   Superstitious responding and frequency matching in the positive bias and gambler's fallacy effects [J].
Boynton, DM .
ORGANIZATIONAL BEHAVIOR AND HUMAN DECISION PROCESSES, 2003, 91 (02) :119-127
[9]   A POWER PRIMER [J].
COHEN, J .
PSYCHOLOGICAL BULLETIN, 1992, 112 (01) :155-159
[10]  
Cohen J., 1988, Statistical power analysis for the behavioural sciences, V2nd