Not true. A strong effect can be meaningful for small n. This is well known from power analysis: https://statisticsbyjim.com/hypothesis-testing/sample-size-p...

I also think middlebrow dismissals are discouraged on HN: http://www.byrnehobart.com/blog/why-are-middlebrow-dismissal...

Under the null hypothesis, a difference this large is still extremely unlikely for a group this small. Sometimes, counterintuitively, you don’t need a huge group.

Yeah, this is a fairly standard size for an early trial.

Trials cost money. Even if this is a lucky outlier and it turns out to be no better than the placebo in a larger trial, having a decently strong positive result can hugely improve your chances of getting the money (and interest) needed to run a larger trial.

But it’s worth realizing this small of a trial size likely means there’s a long road to widespread use

Sure, but at that point the complaint is in either the journalism being misleading (implying it's "coming soon" in a layman's use of "coming soon"), or our outsider view into internal messaging is misunderstanding the claims (if this was written for people familiar with the field, "there's a long way to go" is safe to assume as base knowledge, which outsiders may not have).

tl;dr yeah, but everyone knows that / science journalism is terrible.

The same could be said for a massive trial with a small effect.

I am curious what the proper size group is for such dramatic results to make it publishable? The results are quite dramatic, it is not a slim success.

That’s an aggressive take for someone who hasn’t bothered to consider the effect size when claiming to know statistics better than these authors.

The heart of the middlebrow dismissal is the assumption that the dismisser understands a field/industry better than a person who's spent their entire adult lives studying it.