You can use the freed-up engineer time to write better code which you can also keep for the life of the system though, though. It is not too hard to think of things which you could do in a few hours that would pay for RAM upgrades in perpetuity, even at BCC's relatively small scale. (Say, an A/B test which resulted in a 1% lift on my AdWords landing pages.)

It's precisely BCC's small scale that makes spending engineering time on performance optimizations expensive. At Google's scale it's worth several thousand engineers' time.

At which point, you have several thousand engineers. It's a problem that solves itself, as long as you have some kind of margins.

At this points, throwing engineers becomes throwing money.

Really, you just have to weight the cost of the upgrades against what that time is worth. A week of engineer time is cheap compared to upgrades if your code runs on hundreds of thousands of machines.

Yeah. My point is that amortization vs fixed cost can be a hard analysis to do.

this is true, but over the lifetime of most systems, resource usage is going to go up at some linear-ish rate, no matter how efficient the code is. its not worth the time trying to fight that hard bottom line increase. if things are wildly out of control, sure, fix the code. but if you've been running your app for 5 years on the smallest VPS and you're pushing the need to upgrade, your time is probably best spent elsewhere.