This is a great overview of the kubernetes/self-hosted PaaS landscape, but is also clearly a bit of marketing for Flynn. Flynn looks awesome, but also as/more awesome at a glance from my research and test runs is https://github.com/convox/rack which I got closer to working more quickly than Flynn, though it helped that Convox builds on top of docker-compose.yml files, which we were already using for development.
For better or worse, we're going the full kubernetes route though, and still working out what deployment etc looks like. So far a lot of un-DRY looking YAML files have inspired a spike at a tool to help manage creating and maintaining kubernetes yaml[1], which others reading this are welcome to try and I'd love to hear better alternatives to or get feedback/pull requests/etc. I may push up a branch I'm working on later that partially automates creating deployments from a Procfile, but we discovered having seperate Procfiles may make applying env-variable only changes harder, curious to hear if others have any guidelines on that, it's so easy to edit an ENV var on Heroku we took it for granted.
I'd love to hear about any trouble you ran into while trying Flynn so that we can fix it! Feel free to send me an email: jonathan@flynn.io
Convox is a great pick if you want to use AWS-specific services for your entire stack. Instead of using portable open source components, Convox uses AWS services wherever possible, and acts as a lightweight coordinator to combine them into a platform.
Flynn has no external dependencies on cloud features, so you can run it anywhere, whether that's on your laptop, AWS, Google Cloud, a VM somewhere, or bare metal in a colo or private datacenter. We also include RDS-like highly available database appliances so that your whole stack is portable and you are not locked into a single hosting provider.
edited to add: We didn't know about this post until Bitmatica posted it, this is just a great post from a happy user, not planned Flynn marketing!
Yeah, definitely think you guys are both top notch projects, but the article focused on you guys as the end solution and mentioned so many other things en route, I felt people should see one of the other high profile contenders, though forgot to add it was AWS only (added below). It's been a few months now since I kind of timeboxed an hour or two to play with each of 3 I narrowed down to as most likely to offer the easiest transition off of Heroku without being single node like Dokku (you guys, Convox and Tsuru were the 3 I think I tried, and got furthest with and was most impressed with Convox and Flynn). It wasn't a totally fair comparison, as I think I ran Flynn and Tsuru locally via Vagrant, because I could for both I think, but had to actually use real AWS for Convox... but this was all just because I was hoping to make a case at work that we should try one of these actual PaaS-like solutions rather than go for a non-PaaS solution like Kubernetes, as I knew it would not be a trivial transition, as the articles' author discovered. If I ever have time to try it again I'll try and go a little further and provide feedback if I have any!
Yeah I initially linked the .com then thought people might appreciate seeing the product, rack and its open source repo more :-) Weirdly when I set it up, it looked like with only a small change or two to our existing docker-compose.yml it "just worked", but the funny thing was once it was running I couldn't quickly manage to find the web service exposed anywhere to actually test... but it deployed painlessly and showed up running in several ec2 instances and a load balancer in our AWS console! Oh, that't another point, unlike Flynn, Convox is AWS only, so it's more opinionated/less flexible, but for many people that's not really a downside if you only plan to run it on AWS for now anyway. It's also built by a bunch of former Heroku engineers, including David Dollar (https://github.com/ddollar)
From looking at your KY tool, it sounds like the pain point you're having is that there's a lot of boilerplate repetition around loading configmaps and secrets into your containers, is that correct?
If that's not far off the mark, have you looked at using an object-oriented k8s client? I've been playing with a pattern where I create classes which know how to render themselves into k8s API objects, and that gives the ability to use language-level looping to create, for example, lists of env variable definitions in the resultant spec.
I'm writing python apps so PyKube[1] is the base client that I've used, but there are clients in lots of languages at this point.
I also tried out Helm, which looks like it has a way of generating k8s yaml files using Go templates, and given a set of config parameters[2]. Unfortunately my use case was a bit too dynamic to fit in that mould, but this seems like it could be useful for many things.
Sorry I didn't see this sooner! The repetition is definitely something that bothered me in the early working versions of our app on kubernetes, and also the general lack of reproducibility and manual fiddling that might be required to make changes or create new services, and the difficulty of having the configuration live in the code so it can change on a per branch basis, and have that branch built correctly by CI when it goes to QA/staging.
ky is definitely still just in the "maybe useful/maybe practical" territory, but now that the basic concept of compiling from a Procfile seems to work, I suspect adding some configuration and more environment aware and extensible/overrideable templating could make it pretty practical, but I also had/have my eye on Helm. There may be some practical/logistical complications in using Helm, and I think if we went that way we'd probably go full hog and use Deis Workflow (https://deis.com/workflow/), which also deserved mention in this thread's discussion, but the k8s clients like the python one you point to or the ruby one (https://github.com/abonas/kubeclient) may be worth bringing into the mix depending on how this all works out... Thanks for the links and for having a look!
Cool, thanks for the explanation. Interesting times living on the frontier here, lots of best practices (and the tools to support them) yet to be written. Definitely interested to follow your experiences, keep sharing :)
We've been using Convox too in Production after some time on Heroku and later AWS (Beanstalk and and CodeDeploy + other services) and I must say it is working as a charm for us. For us it's the right balance of control, ownership, ease of deployment and management. Plus, Convox have proved to provide a great service when we needed any extra support. I can't comment about Flynn though.
For better or worse, we're going the full kubernetes route though, and still working out what deployment etc looks like. So far a lot of un-DRY looking YAML files have inspired a spike at a tool to help manage creating and maintaining kubernetes yaml[1], which others reading this are welcome to try and I'd love to hear better alternatives to or get feedback/pull requests/etc. I may push up a branch I'm working on later that partially automates creating deployments from a Procfile, but we discovered having seperate Procfiles may make applying env-variable only changes harder, curious to hear if others have any guidelines on that, it's so easy to edit an ENV var on Heroku we took it for granted.
[1] https://github.com/stellaservice/ky