Ok, I have just reached 100 concurrent containers in Europe, which is max at this stage and I don't think I can change the aws limit on the fly (will check though) so I might start killing the oldest instances.
Apologies for inconvenience folks, but I am sure people rarely play with each instance for the whole 3 hours anyway at this stage.
Honestly did not expect this kind of "public beta" rollout, still surprised how well the service holds so far under the load.
If anybody's curious about this "public beta hosting - HN bear hug" performance load test:
I just (sequentially) restarted all 4 servers behind my 2 load balancers - seems that by reaching 100 containers in each geography the provisioning threads may have died after many (unsuccessful) retries, so need to restart the thread pools.
That is one more error I need to handle (or increasing my limit in AWS).
Heya - just a note that I build these kinds of PaaS platforms for a living - if you need a hand building a super-massive cluster for lots of users, let me know!
Apologies for inconvenience folks, but I am sure people rarely play with each instance for the whole 3 hours anyway at this stage.
Honestly did not expect this kind of "public beta" rollout, still surprised how well the service holds so far under the load.