Ah. I feel a bit stupid for taking that so literally.
So, when i delete a directory, whenever possible, i use the rmdir command. rmdir refuses to delete a directory which isn't empty. It's a really handy safety check. If a colleague dropped a tilde directory on me, the process would go:
Well, i'll just delete that.
$ rmdir ~
rmdir: /home/twic: Directory not empty
Oh bugger, of course. Better quote it.'
$ rmdir '~'
Right, now i will proceed to give my colleague a Chinese burn.
