>>> Every scientist should know how to evaluate their results from a Bayesian perspective, at least qualitatively.
I'm a scientist. How do I do this? I took one stats course, more than 30 years ago, and it was mostly proofs. Of course we learned Bayes' Theorem, and solved some problems using it.
What I don't remember is being taught to choose an "ism" for doing math, so it could easily be something that I'm already doing and taking for granted.
At a minimum, recognize that not all p < 0.05 results have an equal claim on scientific truth. The study power and the prior probability of the hypothesis being true also influence the posterior probability [1]. In particular, underpowered studies of implausible hypotheses provide almost no scientific signal [2], and p < 0.05 is meaningless in such cases.
Check. Thankfully I come from the physics world, where we don't trust ourselves with any analysis method unless we have some independent way to make sure our results make sense. Most physicists are uneasy about "fancy" statistics.
There's the famous quip: "If an experiment needs statistics, then one ought to have done a better experiment." While this is a bit extreme, it does represent a tendency to design experiments, when possible, that lend themselves to relatively basic data analysis. In fact, I haven't ever calculated a p value in my work.
From what I've read about Bayesian statistics, I'm not sure it's going to get the life sciences out of the mess they're in. That's just my personal hunch. The trouble is, for justifiable reasons, life science research is hard, and is conducted with urgency due to the immediate need for solutions (such as vaccines, therapies for mental illness, or cynically, advertising revenue).
A good rule of thumb seems to be: Multiply p by 10. In other words, p < 0.05 is a coin toss. This rule seems to replicate the entire replication crisis. ;-)
I'm a scientist. How do I do this? I took one stats course, more than 30 years ago, and it was mostly proofs. Of course we learned Bayes' Theorem, and solved some problems using it.
What I don't remember is being taught to choose an "ism" for doing math, so it could easily be something that I'm already doing and taking for granted.