So... what do actual real people do in practice? I am very confused what people are actually doing here.
LOTS of people seem to have moved to this kind of docker-based deploy for PaaS. They can't all just be... ignoring security patches?
I am very confused that nobody else seems to think the "handle patches story" is a big barrier to moving from heroku-style to docker-style... that everyone else just moves to docker-style? What are they actually doing to deal with patches?
I admit I don't really understand your solution -- I am not a sysadmin! This is why we deploy to heroku and pay them to take care of it! It is confusing to me that none of the other PaaS competitors to heroku -- including fly.io -- seem to think this is something their customers might want...
I mean, every Dockerfile has a FROM some_base_image line at the beginning, and it refers to an image and an optional tag (which defaults to the `latest` tag if omitted). As far as I know, whenever you build your docker image, it will check for a new version of the base image and download that if needed. So e.g. if you have FROM ubuntu:20.04 at the top of your Dockerfile, and a new version of ubuntu:20.04 is uploaded to dockerhub, the base image will be updated the next time you run docker build, which shouldn't be too long if you're making deployments regularly.
"I admit I don't really understand your solution -- I am not a sysadmin! This is why we deploy to heroku and pay them to take care of it!"
Right. I didn't explain it very well then.
Basically it's a heroku-ish solution but as a tool rather than a service, and where you build locally instead of pushing source code to some remote cloud. You say "here's my build system, go push to these plain Linux VMs". Now, someone needs at least a bit of sysadmin knowledge - you need to know how to obtain Linux VMs, set up access to them, and make sure they're (self-)applying security updates. From time to time you'll need to roll to new OS releases and do restarts for kernel fixes. But that stuff isn't all that hard to learn.
Still - I'd be curious to know where your threshold is for touching Linux. With Heroku you never did, right? Dynos could have run Windows for all you knew? If you had a tool that e.g. you gave your cloud credentials to, and it then spun up N Linux VMs, logged in, configured automatic updates and some basic monitoring then let you push servers straight from your git repo, would you be interested in that? How much did you rely on Heroku support going in and helping you fix app-specific problems live in production?
It seems not that hard to just manually bump your version periodically, right? I get that it's not as nice as having it taken care of for you, but it's not like you're having to manage a whole system by hand
Assume I've never used Docker before, can you tell me what "manually bump your version periodically" means with regard to this question? Can anyone give me concrete instructions for what this looks like in a specific context? Like, say the first Dockerfile suggested in OP?
The Dockerfile has in it:
> ARG RUBY_VERSION=3.2.0
> FROM ruby:$RUBY_VERSION
Which it says gets us "gets us a Linux distribution running Ruby 3.2." (Ubuntu?). The version of that "linux distribution" isn't mentioned in the Dockerfile. How do I "bump" it if there is a security patch effecting it?
If you're basing on a lang-specific container like ruby, then it's the version of that container in the FROM line. Notice how ruby images come in various versions of OS (https://hub.docker.com/_/ruby). You can specify that as part of the FROM string. However, they also let you drop the OS part, and only specify ruby version. This will usually default to an image with the latest OS provided by that docker repo. Nothing to bump in this case, just occasionally rebuild your own image from scratch, instead of from cache, to make sure it downloads the latest base image.
This right here. Most of the time you just need to rebuild your image. If the project is being actively developed and built, nothing to worry about. (Unless you pin to a very specific OS version of course).
If it's not, you just need to trigger a build every so often. Maybe this could be a feature PaaS offers in the future.
Docker caches the results of each command, so to "bump" the versions you have to trigger a rebuild of the whole image and tell it to toss the cache. So it'll re-run apt-get update and use whatever the latest stuff is as of the moment of rebuild.
There are a few problems with this, as noted elsewhere in the thread:
1. You have to do an uncached rebuild and repush all of your images, using some ad-hoc company specific process. There's nothing that can do this for you at the end points or service levels, because Docker images are meant to be immutable after build and don't come with the scripts or inputs used to build them.
2. The default is to use caching, so devs may not notice that they didn't refresh their base OS for a while.
3. You don't get notified when updates are available or applied. There's nothing like the unattended-upgrades package that comes with Debian normally, which will apply upgrades and then tell you what happened.
4. Because of (3) the latency is very high. There is no story (other than third party scanners) for getting notified about urgent upgrades. If there's another zero day in OpenSSL then with a standard Linux install you'll get patched as soon as a new package is released and your machines update, so pretty quick (a day or so). With Docker images, it'll get patched on an app-by-app basis if and when people get around to doing an uncached rebuild and repush of the image.
5. Kernel upgrades are a whole can of eels in container-world. People like to think of the container as being a self-contained OS but it isn't. There is a largely unstated and untested assumption that any Linux distro user space can run on any kernel version or configuration, regardless of whether the OS originally shipped in that configuration, and everything will just automatically do something sensible. Mostly this assumption is OK because servers are very simple, but it's not actually guaranteed by anything. A lot of people misunderstand the "stable Linux syscall interface" guarantees and what that means.
It's for reasons like this that I prefer the slightly older way of running real binaries that are exposed to the OS and which use OS specific packages. I configured unattended-upgrades and use LTS versions of the OS, so that security patches just stream in without me doing anything. There are a few downsides to this too:
1. You have to either restart your servers from time to time to force security patches to actually get reloaded into memory, or use the needrestart Debian package - however that only works if you're using package metadata properly.
2. You do need to understand at least a bit of Linux sysadmin. Enough to know how to ssh in as root, use apt-get and so on.
3. The tooling story is poor, hence my musings above about demand for something better. Without Docker, today you're going to be manually copying files to the server, having to learn systemd and how to start/stop/enable services, how to restart them on upgrades etc. That's why I've written something that does it all for you.
Most places are building software regularly. Once a week update the base layer of the container to the latest OS. Then update your libs and app. Run tests, deploy.
So... what do actual real people do in practice? I am very confused what people are actually doing here.
LOTS of people seem to have moved to this kind of docker-based deploy for PaaS. They can't all just be... ignoring security patches?
I am very confused that nobody else seems to think the "handle patches story" is a big barrier to moving from heroku-style to docker-style... that everyone else just moves to docker-style? What are they actually doing to deal with patches?
I admit I don't really understand your solution -- I am not a sysadmin! This is why we deploy to heroku and pay them to take care of it! It is confusing to me that none of the other PaaS competitors to heroku -- including fly.io -- seem to think this is something their customers might want...