As someone 8 months into a major Nix packaging effort (1200 package definitions, mostly auto-generated), this is 1000% the most significant issue. Reading the pills can get you the first little bit, like the difference between derivation and mkDerivation, but so much later stuff is just completely undiscoverable and inconsistent, like the fact that overrideAttrs may subtly not do what you want if it's a Python package since you have to use overridePythonAttrs instead, but the corresponding function for other builders (eg overrideBazelAttrs) doesn't exist.
This is indeed not very good. Note there is a reason that overridePythonAttrs exists: it overrides the call to buildPythonPackage instead of mkDerivation. There is/was an RFC on standardizing overriding in Nixpkgs but it got stuck. I think for these things to improve what is really necessary is funding to improve Nixpkgs. These kind of issues are fairly hard to solve as they span multiple ecosystems and require coordination.
Oh, of course, and in my case it was an important discovery, because my custom packages were all also defined in terms of my own mkDerivation-wrapping function, so I happily copy-pasted the definition of overridePythonAttrs to make my own overrideMyAttrs version of it.
But it was brutal that I had to just stumble across this. I think the article correctly identifies that there is decent enough documentation in Nix for:
- Beginners to the nix language/concept (Nix Pills)
- Beginner users of NixOS (the NixOS manual)
- Beginners to packaging for Nix (wiki articles, blogs)
But there is a huge, huge gap for the user who is trying to cross the chasm and become a true expert in it all— who wants to understand at a deep level things like how overlays and namespaces work, how bootstrapping works and what is the division of responsibilities between nix and nixpkgs, plus the inner workings of things like buildEnv and stdenv.
I mentioned it already below, but just to emphasize: I think Burke Libbeys playlist on Nix [1] is that missing piece. It lays down a solid foundation of understanding on what Nix is both conceptually and syntactically. You are then able to derive the rest of the needed understanding by clicking through the nix source of whatever it is you are trying to figure out.
As someone who really likes the idea of declarative system configuration and management, shares the vision of nix and guix, and have been dipping my toes by using nix package management on macos (and sharing most of the experiences mentioned upthread):
I do thank you for the pointer, and I think I've come across these videos before, _but_ to me, video is probably the last format I can assimilate this type of information. Usually, in my experience, well designed systems have a way of making sense so that documentation and understanding flows naturally from base abstractions, and this is something I miss in nix. I would like to try guix to see whether that makes more sense to me, but I am hesitant given that nix seems to have so much more of the mindshare.
One day I'll try to give the video tutorials a chance, maybe it will clear things up. Thanks again and thanks to mr Libbey as well.
> well designed systems have a way of making sense so that documentation and understanding flows naturally from base abstractions
Honestly, thats the main revelation I had when going through that playlist. Basically Nix is just attribute sets (which is sort of just a JSON object). The entire thing. Nix files are just some preamble and then they return an attribute set at the end. Understanding that will get you 90% of the way there in terms of understanding the syntax and being able to cut through the "weirdness".
At that point its about understanding actual concepts like overlays, derivations, etc but its much easier now that you understand the attrset concept.
I hear you, I think I understand what you mean and I definitely want to believe you, but how come this can't be explained in a succinct written documentation, instead of a series of video tutorials by example?
I do realize it's a work of passion done by numerous volunteers, to whom I feel gratitude, but at this point, why can't this be adequately defined, in a written form, in a way that is understandable? To me, the concept of a series of projections of declarative configuration statements onto a final state is not hard to understand in itself; yet, I am somewhat struggling to create and maintain such a configuration without feeling that I am just copypasting bits and pieces from random people.
I guess I just have to bite the apple and spend a few days watching YouTube to see if I reach enlightenment - at this point, I barely understand the function names.
Yeah like I said, I struggled for a couple _years_ on having a feeling of complete confusion when copying a shell.nix to every project I spun up. I was never able to understand _any_ of the written documentation. After that playlist I easily click through the actual nixpkgs repo itself and find almost everything I need to handroll my own derivations.
It seems like the community is just a bunch of people who have had that lightbulb go on already, so its hard to go back and view things without the lightbulb on.
The language reference worked out fine for me but the chasm between "basically running" and "knowing what I'm actually doing" has been substantially painful and I'm still not on the other side.
Though I say this while continuing to slog through said pain as I find time because once I've learned a particular thing it's always felt like it was worth the effort to get there.
