I don’t want Kubernetes period. Best decision we’ve made at work is to migrate away from k8s and onto AWS ECS. I just want to deploy containers! DevOps went from something you did when standing up or deploying an application, to an industry-wide jobs program. It’s the TSA of the software world.
where do you keep the ECS service/task specs and how do you mutate them across your stacks?
How long does it take to stand up/decomm a new instance of your software stack?
How do you handle application lifecycle concerns like database backup/restore, migrations/upgrades?
How have you supported developer stories like "I want to test a commit against our infrastructure without interfering with other development"?
I recognize these can all be solved for ECS but I'm curious about the details and how it's going.
I have found Kubernetes most useful when maintaining lots of isolated tenants within limited (cheap) infrastructure, esp when velocity of software and deployments is high and has many stakeholders (customer needs their demo!)
Not sure if this is a rhetorical question but I'll bite :-)
> where do you keep the ECS service/task specs and how do you mutate them across your stacks?
It can be defined in CloudFormation, then use CloudFormation Git sync, some custom pipeline (ie. github actions) or CodePipeline to deploy it from github
You can also use CodeDeploy to deploy from Git or even a AWS supplied github action for deploying a ECS task.
> How long does it take to stand up/decomm a new instance of your software stack?
It really depends on many factors, ECS isn't very fast (I think it's on purpose to prevent thundering herd problems).
> How do you handle application lifecycle concerns like database backup/restore, migrations/upgrades?
From what I learned from AWS is that ECS is a compute service and you shouldn't persist data in ECS.
Run your database in RDS and use the supplied backup/restore functionality
> How have you supported developer stories like "I want to test a commit against our infrastructure without interfering with other development"?
If it's all defined in CloudFormation you can duplicate the whole infrastructure and test your commit there.
Yeah, that doesn't really answer the question at all...
Do you just have a pile of cloudformation on your desktop? point and click? tf?
And then none of the actual questions like
> How do you handle application lifecycle concerns like database backup/restore, migrations/upgrades?
There is no difference between cloudformation, clicking, terraform, boto, awscli, pulumi, or whatever else. The platform at the other end of those tools is still ECS.
Backing up databases isn't the job of the container-running platform (ECS), especially not in AWS-world where databases are managed with RDS.
The rest of the questions were "how do I run containers on ecs?" in various forms. The answers to all of them is "by asking ecs to run containers in various forms."
ECS is very very similar to Kubernetes and duplicates pretty much all of the functionality except AWS names and manages each piece as a separate service/offering.
ECS+Route53+ALB/ELB+EFS+Parameter Store+Secrets Manager+CloudWatch (Metrics, Logs, Events)+VPC+IAM/STS and you're pretty close in functionality.
I'm so confused about the jobs program thing. I'm an infra engineer who has had the title devops for parts of my career. I feel like I've always been desperately needed by teams of software devs that don't want to concern themselves with the gritty reality of actually running software in production. The job kinda sucks but for some reason jives with my brain. I take a huge amount of work and responsibility off the plates of my devs and my work scales well to multiple teams and multiple products.
I've never seen an infra/devops/platform team
not swamped with work and just spinning their tires on random unnecessary projects. We're more expensive on average than devs, harder to hire, and two degrees separated from revenue. We're not a typically overstaffed role.
> These days often DevOps is done by former Software Engineers rather than "old fashioned" Sys admins.
Yes, and the world is a poorer place for it. Google’s SRE model works in part because they have _both_ Ops and SWE backgrounds.
The thing about traditional Ops is, while it may not scale to Google levels, it does scale quite well to the level most companies need, _and_ along the way, it forces people to learn how computers and systems work to a modicum of depth. If you’re having to ssh into a box to see why a process is dying, you’re going to learn something about that process, systemd, etc. If you drag the dev along with you to fix it, now two people have learned cross-areas.
If everything is in a container, and there’s an orchestrator silently replacing dying pods, that no longer needs to exist.
To be clear, I _love_ K8s. I run it at home, and have used it professionally at multiple jobs. What I don’t like is how it (and every other abstraction) have made it such that “infra” people haven’t the slightest clue how infra actually operates, and if you sat them down in front of an empty, physical server, they’d have no idea how to bootstrap Linux on it.
Yeah, DevOps was a culture not a job title, and then we let us software engineers in who just want to throw something into prod and go home on friday night, so they decided it was a task, and the lowest importance thing possible, but simultaniously, the devops/sre/prod eng teams needed to be perfect, because its prod.
it is a wierd dichotomy I have seem, and it is getting worse. We let teams have access to argo manifiests, and helm charts, and even let them do custom in repo charts.
not one team in the last year has actually gone and looked at k8s docs to figure out how to do basic shit, they just dump questions into channels, and soak up time from people explaining the basics of the system their software runs on.
Thats great if that works for you, and for a lot people and teams. You have just shifted the complexity of networking, storage, firewalling, IP management, L7 proxying to AWS, but hey, you do have click ops there.
> DevOps went from something you did when standing up or deploying an application, to an industry-wide jobs program. It’s the TSA of the software world.
DevOps was never a job title, or process, it was a way of working, that went beyond yeeting to prod, and ignoring it.
From that one line, you never did devops - you did dev, with some deployment tools (that someone else wrote?)
I mean you can edit a big YAML file inside ArgoCD, but what I'm building is an actual web form (e.x. `spec.rules[].http.paths[].pathType` is a dropdown of `Prefix`, `ImplementationSpecific`, `Exact`), and all your documentation inline as you're editing.
People have tried this before but usually the UI version is not fully complete so you have to drop to YAML. Now that the spec is good enough it's possible to build a complete UI for this.
Yup, and it has the advantage of having a easily backed up state store to represent the actions of the GUI.
I always liked the octant UI autogeneration for CRDs and the way it just parsed things correctly from the beginning, if they had an edit mode that would be perfect
Is there anything in particular you like about what Octant does? I don't see anything that actually looks at the object spec, just the status fields / etc.
