You build a metaoptimizer. The metaoptimizer decides which new layers are necessary, and whether the cost of "buying new neurons" ( adding compute instances & storage ) is worth the benefits to the high level optimization functions. ( or pruning / reallocating ). Meta-optimization is clearly a thing, it's in the research papers, and once you build a good one, you're done programming.

My new prediction is Skynet will come out of the financial bots. It solves the money problem, because those bots will become financially self-sufficient quickly.

Regrettably, the target function is "make money" which will cause our next-gen species to be psychopathic and uninterested in human life, except, in the short term, as a market to be plundered. Not a good outcome.