Hacker News new | past | comments | ask | show | jobs | submit login

But in the real world, we do less rigorous studies first and follow up with more rigorous studies if our preliminary investigations show promise.



I always wonder if this is a good idea. While getting a false positive is not really a problem, because you're going to do a follow up experiment, what happens to the things we miss? If you do an experiment that doesn't really have a large enough sample size, or comes from a biased sample (because it's really an offshoot of a different experiment) and you decided that there is no effect, does it stop others from researching that effect? I suppose since we don't tend to publish negative results maybe it doesn't matter, but it's always something that has niggled at me.


The trade-off between Type I and Type II error is an inherent problem in research. But false positives are most certainly a problem, too. Just look at the issues psychology and biomedicine have been grappling with in terms of replication. Whole careers were wasted based on what seem now like false positives.




Consider applying for YC's Spring batch! Applications are open till Feb 11.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: