In this case I had to use rescue mode. As even different tty was too slow to respond.

Also just removing some stuff up wasn't working.

Sounds like swapping/trashing + an exception loop taking all resources

Could it depend on what file system you use, and OS settings you have?

For example, Google says that by default ext4 will reserve 5% of disk space.

So depending on your system, a “100% full” drive might still have a (sometimes significant) bit of space left.

> For example, Google says that by default ext4 will reserve 5% of disk space.

Reserved 5% for root, but if the files/logs that are taking up space are written by the root user then it's a moot point:

> Specify the percentage of the file system blocks reserved for the super-user. This avoids fragmentation, and allows root-owned daemons, such as syslogd(8), to continue to function correctly after non-privileged processes are prevented from writing to the file system. The default percentage is 5%.

* https://man.archlinux.org/man/mke2fs.8

One trick possible with ZFS is to take a pool (e.g., rpool) and create an 'extra' data set in it (rpool/reservation) and set a reservation for it, so that even if all the other areas are filled (rpool/root, rpool/var, rpool/home) you can simply lower the reservation to get some space back quickly.

> if the files/logs that are taking up space are written by the root user then it's a moot point

Yeah. Sorry if my comment was a bit unclear but what I meant to say is that because of this reserved amount I could see it being the case that one machine is still able to finish booting even though the hard drive is “full”, because it is using the reserve capacity to allow processes to write to disk.

Whereas the other person that was unable to boot may have either been using a different file system without reserved amounts, or have settings that made it so reserve was set to 0, or their machine could have been actually physically out of space.

Most people turn that off coz it is near-useless feature and I don't need OS reserving 5% of 2TB drive

In general both ext3 and ext4.

If you use SSH agent, key-based auth will fail and fallback to password, because that requires creating file in /tmp that you can't do if partition with ssh ran out of space.