It’s not that clearcut, because sometimes you forget all the packages you installed manually over the years and aren’t using anymore, and it can make sense to be offered to remove one when it helps resolving a conflict with a newer package that you actually want to use now.
I’ve been upgrading Debian in-place for almost two decades now and prefer that approach. Security updates are automated (daily), major-version upgrades have almost no downtime, and only very rarely is there a critical configuration that needs to be adjusted.
The installed packages can be listed with `dpkg --get-selections`, and that list can be replayed should it be necessary to recreate the installation, plus a backup of /etc/. But I never had to do this, Debian just works.
I was previously a maintainer of certain Debian packages, and of similar vintage, so this advice comes with the extra salt of having seen how the sausage is made. I shudder to think how many abandoned files, orphan packages, and obsolete/bad-practice configurations might be lurking in a system that has only been release-upgraded for decades. Yes, no doubt it functions. By the same token, people can live in their own filth. Should they? I choose not to.
That said, I may do a speculative dist-upgrade on a snapshot to reveal & prepare for conflicted conffiles in advance, but I'll throw that away, I won't rely on the merged result across a release upgrade.
Debian releases are somewhat slower than Ubuntu or similar - given infinite time, esoteric configurations will break on update due to some edge case 4 dist-upgrades ago.
> The healthy choice is to rebuild rather than upgrade any long-lived OS installation (e.g. your desktop) on distro release.
I wish there was more rigor (and testing) with this sort of thing. Generally systems should have the invariant that “install old” + “upgrade” == “install new”. This property can be fuzz tested with a bit of work - just make an automated distribution installer, install random sets of packages (or all of them), upgrade and see if the files all match.
/etc makes this harder, since you’d want to migrate old configuration files rather than resetting configuration to the defaults. But I feel like this should be way more reliable in general. And people want that reliability - if the popularity of docker and nix are anything to go by.
Alas that many packages will not upgrade conffiles gracefully when their structure/semantics change, and many are beyond the ability of any package manager's built-in diffing tools to apprehend. That is why I place /etc under scm for long-lived hosts, because what happens next instead looks an awful lot like doing a conflicted three-way code merge following a breaking-change vendor release. The greatest OMFGFFS in this realm, by far, came with the big lurch over to systemd.
> The greatest OMFGFFS in this realm, by far, came with the big lurch over to systemd.
I think by the time I started with it, NixOS was already on systems, but reportedly that transition went off without a hitch. Before the transition, NixOS configs were in charge of generating OpenRC scripts or whatever, then afterwards the same configs generated systemd units instead. When the system has total control over the config files like that you get to bypass certain challenges!
The problem is that's not really true. i.e. consider you get postfixed pulled in by postfix | mail-transport-agent and the new distro release changes it to exim4 | mail-transport-agent as we picked a new default MTA.
Does this mean we should, by default, replace your MTA and require you to have to configure a different one?
You will be able to do this (pick the same automatically installed packages given the same set of manually installed ones; installing missing Recommends, etc, by using --fix-policy and some more options perhaps).
As for configuration this is worse, files managed by ucf have 3-way merging but it's not automatic and most configuration isn't managed by ucf but by dpkg.
In theory yes, but this hits with the fact that defaults change.
For example a new system won't have pulseaudio, but it won't be removed and replaced automatically because that would be potentially disruptive to existing users.
The even healthier choice is to store all your system state in a config, you can then just occasionally clean up the config. This also lets you put extensive comments on why a package is needed or even create a submodule for a specific project.
Shame that the only bistro that lets you do this is extremely arcane and difficult to learn if you do not have a PhD in mathematics.
Is the mapping from config -> state reproducible? Traditional distros drift over time with lots of upgrades because the state is not just a pure function of the set of packages to be installed (the "config"). It could depend on the system's history, such what packages were installed or removed in the past.
The so-called "atomic" distros don't attempt to make the (config -> state) transformation reproducible but rather sidestep the problem by snapshotting and replicating the output of the transformation -- the files on disk after various installs, upgrades, and config changes.
Flakey Windows taught me this doctrine, and it has carried over to any OS that I use (mostly Linux). A directory with 'portable apps' was essential in Windows, fortunately apt makes this a whole lot simpler :)
1. Find ways to automate setting up my env. Keep that instruction current. apt and chocolatey/ninite have made this far simpler than it used to be.
2. Keep data in directories that are auto-backupped. Concretely I have everything in 'shares', and these shares have survived various changes in tooling, buut their location and principle have stayed the same. Also helps organizing your files :)
3. Have short but precise instructions on how to setup new envs, wherever automation isn't possible or impractical.
Has worked for me for decades now, and it means I'm good to go within an hour of (re)installing a fresh OS.
This is one of my favourite things about Fedora Silverblue and rpm-ostree. If I run `rpm-ostree status`, I get a list of all the packages I’ve installed or removed compared to the base image. Makes it really easy to keep track of what’s different in my system compared to the current default.
I don’t know if other distributions do this, but I quite like gentoo’s “world” file. Its a text file listing all the manually installed packages. Everything in the list is pinned in the dependency tree, and kept up to date when you “emerge update”. I constantly install random stuff for random projects then forget about it. The world list lets me easily do spring cleaning. I’ll scroll through the list, delete everything I don’t recognise and let the package manager auto remove unused stuff.
I think nix has something similar. I wish Debian/ubuntu had something like that - maybe it does - but I’ve never figured it out.
I think OSes really should take inspiration from the newest generation of programming languages here. Rust and Julia (as well as a few others) do a lot better than the OS. Features like environments (e.g. for installing per user packages), and a clean separation of a project and an environment would be very nice.
I hesitate to say it because it comes up in approximately every single conversation about package management, but... NixOS does all of those things. Whole list of packages in configuration.nix, whole of /etc encoded into configuration.nix, flakes let you pin everything, (therefore) it's trivial to check the whole OS config (packages, versions, configurations) into version control, home manager lets you do all this per-user (or you can do it in the system-wide configuration.nix if you really want), and you can drop a flake.nix+flake.lock in any project directory and scope things that way.
apt-mark showmanual shows you all manually installed packages and apt-mark auto marks stuff as automatically installed.
The list of manually installed packages is effectively the "world".
The new solver takes inspiration from apk in that it marks the world/manually installed packages the root of the dependency problem (hence why they can't be removed by the solver itself, only manually).
Yeah, I've hit that exact situation on one of my machines running Void Linux (so obviously completely different package manager). I personally think it's best to tell the user that it can't make the given constraints work and suggest which package(s) are the problem, and then if the user is willing to part with them then they can `apt remove foo && apt upgrade` or w/e (which is more or less my experience of how Void's xbps does it).
