The ancients were misled by apparent size of stars, not realizing they were point sources spread out by atmospheric diffraction. Placed far enough away, they would have to be implausably big by that measurement.

So people promoting distant stars had a glaring contradiction they had to ignore. Lesson for modern theorizing.