Similar arguments were made during our agrarian period, there's only so much farmable land so there is a limit on population and growth. Productivity gains and innovation remove that issue.
Surely sooner or later as we run out of resources (and destroy the planet enough) we have to move away from the idea of growth and toward sustainability. These are mutually exclusive goals.