A problem less easily confronted is known as the "file-drawer effect." Researchers generally published only their positive results and consigned the rest of their data to their files. Statistically speaking, this amounted to reporting only the flukes, the improbable results that do sometimes crop up, just as 7 may, very infrequently, turn up five times in a row at the craps table.

There's also what academics call "the file-drawer effect." "If you do a study that shows no differences, you assume it won't be published," says Claire Etaugh, professor of psychology at Bradley University in Peoria, Ill. "And until recently, you'd be right. So you just file it away."

For any given research area, one cannot tell how many studies have been conducted but never reported. The extreme view of the "file drawer problem" is that journals are filled with the 5% of the studies that show Type I errors, while the file drawers are filled with the 95% of the studies that show nonsignificant results.