> So IMO, the first reform to conduct is to turn clinical researchers into real scientists. That won't be easy.

Mostly, there are not enough hours in a life to be both a good doctor and a good scientist. The few MD-PhDs who give equal weight to both areas are both brilliant and extremely driven. Multidisciplinary teams seem to have a better chance at success.

I guess if the scientific education was done early (undergraduate at the latest) dual training could work. Once medical training and practice starts there isn't a lot of time left over. And early / non-practicing science education, for whatever reason, doesn't seem to be very effective.

I was told long ago that the first rule of evaluating a medical treatment is to ignore all the clinical studies because nothing useful comes out of them.

I don’t know if that’s actually the case but that seems to be the prevailing thought. Which is awful, because it should be the opposite. Clinical studies should be the gold standard. They’re often conducted by people who are better doctors than scientists though. A zero-blind trial of n=4 might perhaps not be the definitive study the doctor had hoped.

(Not to pick on doctors. My wife’s one. Software developers often fall into the same common “I’m good at this one hard in-demand skill so I must be good at all of them” trap.)

To ignore _all_ clinical studies because “nothing useful comes out of them” seems extreme. Yes, there are poorly run studies, and conclusions can be inappropriately drawn from studies (ex: when a study wasn’t meant to answer certain questions).

However, well-run randomized control trials (RCTs) in medicine absolutely exist and can be a very strong form of evidence when interpreted correctly. This is especially true in comparison to other disciplines (economics, etc.) where such RCT trials are impossible and researchers wish that they a research tool as valuable as an RCT.

Fair. It was mainly in the context of “clinical studies show that unlikely thing X cures Y”. No randomized double-blind trials? Sure.

Interested to hear what you mean when you say "turn clinical researchers into real scientists." Are you thinking of expanding the MSTP? Or just better statistical literacy for physicians? Or something else entirely?

I don't know of any magic formula. We'd have to force the change somehow, because it won't correct itself. But the path is unclear, if there is a path at all. The current situation is that most clinical research is performed by MDs/MD-PhDs with no scientific background else than what is taught in medical/doctoral school, i.e. they lack the basics, and are usually more interested about advancing their careers rather than science. That said, systems shape people, and not the other way around.

I am an MD/PhD, and I agree that a lot of the research done by clinicians is self-serving and useless. But most medical/health research is not done by physicians who are typically quite busy and generally not motivated to do anything that involves statistics and takes time and effort. Also, most clinician-led research is simply ignored because it tends to be overridden by the results of large clinical trials usually conducted by professional scientists. Where I see major problems are in two fields. The first is epidemiology (which receives about 30% of Research funding) where most of the research is done by non-clinicians, often involves data dredging and posthoc theorizing motivated by easy access to larger amount of pre-collected data and the availability of statistical software that permits fitting black box models that typically show whatever results are good for publication. This field is the source of contradictory results that we read every day in the news where something and its opposite are both good and bad for every kind of imaginable illness or symptom -. The other is a lot of basic sciences research which typically represent about 60% of all funded research. The great majority of that research is baseless as was shown by effort to replicate many of the findings. Billions of dollars are spent searching for genetic and other causes of illness using small poorly designed studies that can neither prove or disprove anything. So the problem is much bigger than just poorly trained and busy clinicians trying to prove a pet hypothesis. The problem is everywhere. Edited to fix typos

I also agree with what you wrote here. But I have much more experience with clinical research, and know less about other fields.