It's actually not as hard as you think for experimental design to show strong causitive effects - I think randomized controlled trials (in animal and then human models) and meta-analysis are much more effective than epidemiological studies (which don't have effective controls, often don't appropriately correct for other factors, or are specific enough to target specific lines of inquiry and allow spurious conclusions to be drawn).

I've just started doing my own research digging through the literature in PubMed and there's a huge amount of research over the past decade especially that's really advanced our understanding of nutrition and metabolism (that sadly, has yet to get full traction in conventional wisdom/popular science).

Just a few examples from my collection on fasting (at this point I'm tracking hundreds of publications covering a whole range of different aspects of nutritional science, and this is only over the course of a couple weeks of research):

Nørrelund, Helene, K. Sreekumaran Nair, Jens Otto Lunde Jørgensen, Jens Sandahl Christiansen, and Niels Møller. “The Protein-Retaining Effects of Growth Hormone During Fasting Involve Inhibition of Muscle-Protein Breakdown.” Diabetes 50, no. 1 (January 1, 2001): 96–104. https://doi.org/10.2337/diabetes.50.1.96.

Bhutani, Surabhi, Monica C. Klempel, Reed A. Berger, and Krista A. Varady. “Improvements in Coronary Heart Disease Risk Indicators by Alternate-Day Fasting Involve Adipose Tissue Modulations.” Obesity (Silver Spring, Md.) 18, no. 11 (November 2010): 2152–59. https://doi.org/10.1038/oby.2010.54.

Harvie, Michelle N., Mary Pegington, Mark P. Mattson, Jan Frystyk, Bernice Dillon, Gareth Evans, Jack Cuzick, et al. “The Effects of Intermittent or Continuous Energy Restriction on Weight Loss and Metabolic Disease Risk Markers: A Randomised Trial in Young Overweight Women.” International Journal of Obesity (2005) 35, no. 5 (May 2011): 714–27. https://doi.org/10.1038/ijo.2010.171.

Chaix, Amandine, Amir Zarrinpar, Phuong Miu, and Satchidananda Panda. “Time-Restricted Feeding Is a Preventative and Therapeutic Intervention against Diverse Nutritional Challenges.” Cell Metabolism 20, no. 6 (December 2, 2014): 991–1005. https://doi.org/10.1016/j.cmet.2014.11.001.

Rothschild, Jeff, Kristin K. Hoddy, Pera Jambazian, and Krista A. Varady. “Time-Restricted Feeding and Risk of Metabolic Disease: A Review of Human and Animal Studies.” Nutrition Reviews 72, no. 5 (May 1, 2014): 308–18. https://doi.org/10.1111/nure.12104.

Fontana, Luigi, and Linda Partridge. “Promoting Health and Longevity through Diet: From Model Organisms to Humans.” Cell 161, no. 1 (March 26, 2015): 106–18. https://doi.org/10.1016/j.cell.2015.02.020.

I should add that the evidence was so strong for certain behaviors that I have changed my diet (VLCHF + 16+:8IF) and n=1 shows immediate improvements in weight, energy, mental clarity, and even skin health. Somewhat unexpected because I wasn't eating that badly according to common wisdom (and have maintained a moderate overweight, but stable weight for the past 3+yrs).

I've been doing this for more than 17 years. Some thoughts:

1. Anything that makes a real change will have side effects. Some of those side effects can be negative. For people not familiar with the concept of a healing crisis, this can be a confusing fact. They can end up temporarily feeling worse and conclude this is a bad thing rather than a temporary stage of a process headed in the right direction.

2. Prescribed medicine typically has a long list of provisos and side effects. If you have a serious condition, prescribed medicine trades short term gains for long term costs. Doctors claim credit for the short term gains, then blame the long term costs on your condition or age rather than on long term drug use. You need to be leery of the possibility that you are doing the same sort of thing and crediting any positive changes to this dietary change and dismissing any negative changes as coincidental or due to something else.

3. I've seen people post questions to a discussion forum along the lines of "What are non drug alternatives for treating X issue? I would like to participate in a drug trial and they won't accept patients taking this drug. I need to get off it to qualify." So people doing stuff like that are confounding the results of the trial and will report changes in symptoms and attribute them to the drug being tried rather than attributing them to getting off some other drug and using non drug alternatives.

4. In practice, it's quite hard to isolate one specific dietary factor because dietary changes typically involve at least two factors. If you add some new food to your diet and eat the same calories, it displaces some other food. You can't say for certain if the changes are due to adding the new food or removing what was replaced. If you are experimenting with your own diet, it helps to start with supplements, make one and only one change at a time and track things somehow. This makes it possible to get some idea of what effect X is having.

5. It's a moving target. If you are calcium deficient and add calcium to your diet, this may be exactly what you need to start feeling better. Initially, more calcium may be better. If your mental model boils down to "more calcium is better," you may wind up resolving the deficiency and then enter a situation where you are actually consuming too much calcium. This can lead to problems that you may not attribute to consuming high levels of calcium because you already decided "more calcium is better." You may develop new issues and utterly fail to relate them to consuming high levels of calcium.

6. People have a really hard time mentally modeling the road not taken. It isn't completely impossible, but people tend to be bad at it. If you have a specific diagnosis and you know the usual course of the condition, you can compare your health to the typical outcome and make some inferences, but it isn't absolutely conclusive. This is why twin studies are valued: We can't make two copies of you, but if there are already two people with identical genes, that's the closest we get to that.

7. Life is chemistry and there can be myriad other factors being overlooked. I try hard to track not only diet, but how much I walk, environmental factors, etc. I've done this for years and I still find myself blindsided at times and struggling to pinpoint why X is happening currently.

Most people aren't going to be able to read hundreds of studies. If you can and that gives you a mental model that makes sense to you, awesome. That doesn't actually strike me as a convincing rebuttal. It strikes me as more evidence that it's actually rather challenging to figure out dietary stuff. You won't solve it by reading an article or two.

I agree that it's really complex and not straightforward, especially since there are so many unknowns, confounding factors, and since so much is different from person to person (microbiota, genetic and especially epigenetic factors, the effects of hormesis, etc) but at the same time, just because it's complex doesn't mean that science (or individuals) should throw their hands up and say "well, there are no conclusions that can be made about human diet" as that seems to be patently untrue.

Also, just because people can't read hundreds of studies doesn't mean that they aren't out there, or that haven't been done (tens of thousands of studies, of all kinds, which taken together do point to much more specific things than "eat some nuts"), or that it's excusable when medical professionals make recommendations that are contradicted by the preponderence of scientific evidence.