Really? Our 95th percentile only goes above 1s when we are having problems, and nobody with any power in the company thinks that's good enough. Think about how much hardware capacity you need for a site getting even 100s of requests per second. If you can halve the p95 you can decommission or re-allocate close to half of your servers.
As several other people on HN have pointed out more eloquently, it's the variability that kills you faster than the average throughput.
The 100ms was not about end-user response times, it's referring to internal response times between servers. To make a page in 1 second you can't have 3 different services taking 700ms to respond, even if you can make all three calls in parallel. And if you have to call a bunch sequentially, you need the 75th or even the 95th percentile for those services to be pretty good otherwise your 95th percentile for the entire interaction will be very spiky.
As several other people on HN have pointed out more eloquently, it's the variability that kills you faster than the average throughput.
The 100ms was not about end-user response times, it's referring to internal response times between servers. To make a page in 1 second you can't have 3 different services taking 700ms to respond, even if you can make all three calls in parallel. And if you have to call a bunch sequentially, you need the 75th or even the 95th percentile for those services to be pretty good otherwise your 95th percentile for the entire interaction will be very spiky.