Your point seems predicated upon more people being “poorer” rather than focusing on the fact that there wasn’t nearly the wealth inequality we have now (which you mention, but dismiss). Personally, I’m of the opinion that “poor” is relative, and the current age of hyperfocusing on payment is only contributing to all that is bad in the world today. Progress, too, didn’t slow down because the people driving it weren’t paid enough.
...but progress was slower. Exponential growth means it grows faster now than it did then.
And yes, relative poverty matters greatly, but absolute poverty is a real thing even in the US. In 1964, the US launched the war on poverty, and poverty declined. Since the 1980s, some of that progress was reversed, but it's still far better than 1964. (And poverty now is relative, and includes very little literal starvation. That certainly wasn't true in the US in 1964, much less earlier - especially before WWII.