False positive rates of randomized Phase II designs

The randomized Phase Il design for the purpose of selecting a treatment for eventual Phase III testing has recently gained popularity in cancer clinical research. Unfortunately, along with its wider use also come frequent misapplications. The major misuse of the design is the treatment of the Phase II results as ends in themselves without further, definitive evaluation. For binary and censored exponential survival data, we quantify the chance of observing "impressive" between-group differences when underlying distributions are exactly the same in 2-, 3-, and 4-arm selection designs. Depending on one's view of what is impressive, the "false-positive" rates range from 20% to over 40%. We stress that randomized Phase II results are pilots to Phase III evaluations. One should not regard them as conclusive. We caution especially against the inclusion of control arms in such designs because of the propensity for erroneous inferences. We also discuss the inappropriate practice of performing post-hoc hypothesis testing and presenting p-values that are less than 0.05. Control Clin Trials 1999;20:343-352 (C) Elsevier Science Lnc. 1999.