Sadly, this very much mirrors my experience. I love Azure and ASP.NET MVC4. But the moment my deploys went from 5 minutes to 20 minutes and more, I started hating myself for any random code or test I wish I cleaned up first.
Overall though, Azure is a great product and once you wire things up correctly, it makes out to be a fantastic hosting service. I do not have experience with Amazon's hosting stack, but I plan on sticking with Azure for the long haul.
The wiring up is killing it for me - the wrong parts have been overengineered (MVC4, JSON) and not enough attention has been paid attention to the things that need to be stable (Azure dev tools). Overall the service has amazing potential, insofar as you can do a lot more than you can do on something like App Engine (which only supports HTTP - which after using Azure, was a good decision). But it drops the ball on all the basic features, whether they be buggy or missing.
The big issue I have with MVC4 (and many other Microsoft things with which I've been working) is that MVC4 doesn't work to fit around me - I need to work to fit around it. Compare that to working with Flask - supporters say it's a "beautiful API", but really what it comes down to is that Flask is minimal enough (not minimalist), gives you everything you need to work with the web, and if you need to do something complex, doesn't get in the way. It's at the appropriate level of abstraction. MVC4 is not.
I guess we'll see how the cost goes over time as well - not sure how that will pan out. I wish I had better stuff to say, because I was quite excited about it when I first started using it, but I have soured on it and won't use it for anything not Microsoft-centric.
Overall though, Azure is a great product and once you wire things up correctly, it makes out to be a fantastic hosting service. I do not have experience with Amazon's hosting stack, but I plan on sticking with Azure for the long haul.