I helped admin a site in 2001 that did tens of thousands of hps to dynamic content on a mod_perl app. And Bitly's doing static content (301's) of what, a 500 byte payload? Our static site layer did hundreds of thousands of hps with minimum a half meg of payload.
The guy's got a valid point. Bitly's using way overcomplicated tech and employs way more engineers than you need to host a trivial amount of traffic for a large scale site. This is a textbook case of over-engineering.
It wouldn't surprise me if that was a standing order from the top - make sure it can scale to everyone on the planet. I built a service not too long ago where they were expecting 50M users in the first year. Real numbers? It's hovering around 300k users. But the good news is it can handle those 50M if they ever come :-) It is highly over-engineered.
Depends on the flexibility you want and your product's strategy. Bitly could do the same thing with objects in S3, served by Cloudfront, and get their analytics async using Cloudfront's logs. 2 engineers minimum, 3 for good measure. But that doesn't leave you much room to go anywhere else with your business.
