I think that may not always be true though. Especially for highly variable workloads.
How many requests per seconds does your site handle on average vs how many can it handle? What about spikes?
If your previous peak is at 10 rps and you can probably handle 20 rps, when should you start dropping connections to keep the site up?
Most systems are designed to have small margins, for cost purposes, so I imagine this sort of situation might come up quite a bit.
I think that may not always be true though. Especially for highly variable workloads.
How many requests per seconds does your site handle on average vs how many can it handle? What about spikes?
If your previous peak is at 10 rps and you can probably handle 20 rps, when should you start dropping connections to keep the site up?
Most systems are designed to have small margins, for cost purposes, so I imagine this sort of situation might come up quite a bit.