Perhaps in contrast to most people's experiences, I haven't found Nix hard to learn. I think it really helps if you have learned Haskell before. Nix (the language) is really a simple functional programming language, but Nix relies quite heavily on functional programming concepts like laziness, fixed points, etc. A lot of things are underdocumented, but if you understand the Nix language well, it isn't hard to look up definitions in nixpkgs. I can understand that it is all very alien and overwhelming if you do not have a grounding in functional programming.
The primary issue for me has been that Nix is a very deep rabbit hole. You can spend enormous amounts of time on making your configuration more functional and declarative. Pretty much like you can spend enormous amounts of time on customizing an Emacs configuration. It's hard to strike a balance. And outside declaratively defined infrastructure (servers), it's probably not really worth it. I could almost fully reproduce my NixOS system (there is always some mutable state left) with a single command. But takes many months of effort to get to that point. On the other hand, I can set up a fresh macOS or Fedora Silverblue systems with all my customizations in 1 or 2 hours and have to do that maybe once of twice a year? So, ¯\_(ツ)_/¯.
I think the balance is different when you manage a lot of servers and most servers can be defined as a function with a small number of varying parameters.
The other part of the rabbit hole is that software breaks frequently in Nix. Upstreams do not really develop things under the assumption of a non-FHS, immutable system. So, you were going to work on something, but before you know it is 30 minutes later because you ended up fixing some package you need and that broke. Similarly, you'll end up packaging a lot of stuff and spending quite some time making it fit the Nix mold (looking at you Python packages that mutate in-place, sigh).
I love Nix as a principle -- it's declarative, immutable, pure, reproducible. But in practice, you can reap many of the same benefits from impure, inferior alternatives, with far less work. Yes, Docker is an ugly duck compared to Nix, but it brings 90% of the reproducibility benefits and probably everyone on your team can be up and running in hours. Rust's cargo doesn't allow you to specify every non-Rust dependency exactly, but with a Cargo.lock file you can make most of your build reproducible. NixOS is a clean, immutable, declarative system, but other systems offer a subset of its features, such as atomic upgrades/rollbacks, immutable root, and isolated applications (e.g. Fedora SilverBlue + Flatpaks, Fedora IoT, macOS). These alternatives are far more familiar and easier to work with. You can get most (but not all) of the benefits of NixOS with far less work.
I know Haskell, regularly write in rust for fun, and nix is by far the most complicated thing I've tried to learn hahaha. Glad you were able to figure it out, and seriously thank you for sharing your opinion of how you view it even after learning it.
I see some younger folks really investing time in it and it worries me. Sometimes smart people fall under this fallacy of "you have to be smart to learn this so it must be good because it's hard" when really it looks more like quicksand than asphalt. Quicksand is hard to get out of, but there's still no reward for falling in.
I recommend looking at already existing package descriptions for programs written in the same language you are looking for. Chances are, you can hit it enough times to make it package up your program as well.
I don’t think learning nix from its internals is worth it, especially not for beginners.
> I don’t think learning nix from its internals is worth it, especially not for beginners.
Nitpick: what you're calling 'Nix internals' are actually Nixpkgs internals.
FWIW, I think that to some extent this is just the way you have to go when you're trying to quickly take up a new (to you) configuration language for practical purposes.
I've somewhat recently started putting together some simple CI/CD pipelines at work with a tool called Dagger, which is a container image builder-and-runner based on BuildKit and the CUE language.
When I first started learning Nix years ago, I had a lot of packaging trouble with some quirky software that wasn't yet packaged for Nix, and I only ended up getting unstuck by asking for help on IRC. One generous person was hitting me with incredibly helpful links to examples from Nixpkgs and the documentation left and right, and I was astonished with their fluency. I asked them how they knew all this, and they basically said:
> Idk, one day I just sat down and started to read through some of the Nixpkgs codebase, and now I `grep -R` through it a lot to look for examples.
At the time it sounded adventurous and crazy to me, as I was pretty intimidated by the prospect. Eventually that became one of my standard tools: I treat the Nixpkgs repo as another source of documentation.
Fast forward to today and my Dagger implementation efforts, and things got immensely easier for me the moment I started treating Dagger like Nix. I cloned the upstream Dagger repository and started searching through the unit tests with `rg`, and all the answers to my questions about what functionality Dagger exposes via CUE modules were right there. Tons of the guesswork disappeared, even though I don't yet have a great grasp of the CUE language.
In some ways, the Dagger case is harder, though. In the name of simplicity and partly for other language design reasons, CUE is emphatically not Turing-complete. This is interesting and attractive in some ways, but one of the consequences is that sometimes you hit bottom in the CUE libraries to find some compiler directive that links in functionality which had to be implemented in a Real™ programming language (in this case, Go). So to really understand, e.g., the custom data types Dagger uses for secrets, you have to go read code in another language.
By contrast, in the Nix case, the language is powerful enough that you can learn everything you need to for even very advanced Nix usage only by reading Nixlang, either in Nixpkgs or in other examples. You never actually have to dive into true 'Nix internals' (the Nix repo, which is C++ code) to figure out how something in a Nix example is achieved. With a more restricted DSL, you sometimes do.
Maybe one valuable 'missing manual' would be something like 'How to read Nixpkgs'! It's such a rich source of examples. I think making earlier reference to it could help a lot of people make faster progress and feel more effective when learning Nix.
I would love to see a "guided tour of nixpkgs," using real-world packages as examples and building up in complexity as you go.
I feel like so many of the problem I encounter as a new nix user have been neatly solved already and there are probably tons of great examples in nixpkgs, but I have no clue where to look, or even what to grep for.
It would be super helpful if there were a resource with examples of common problems and how they're solved in the real world. For example, say you're packaging something whose unit tests hit the network. There could be a link to a package definition in nixpkgs that overrides the check phase to disable the problematic test.
When anything hits the network, there is nothing to do besides
- disabling the code that hits the network, whether by patching or by configuration ("--disable-downloads" or anything similar)
- or emulating it; e.g. if the script downloads a file, we can `fetchurl` it and move the downloaded file to its expected place. 9Of course, it works when the script does not override the downloaded file; otherwise, go to the previous option)
Two examples came to my mind:
- Arcan vendoring required patching its cmake scripts, because cmake does not honor the cache and tries to download things anyway:
I think the first step to solving problems is acknowledging them: lazy? cool. duck-typed? cool. weird thunk stack traces, eh, workable.
Lazy and pure and duck-typed and broken ass stack traces? Fuck that. Fuck that in particular.
Eelco fucked up. Badly. nixpkgs uses Haskell type signatures as comments in the core ‘lib’s, because duck-typed pure lazy functional is fucking insane.
I agree - Nix, the language, really is at the root of many of Nix' problems.
The bad error messages, the useless stack traces, and the overall un-debuggability of Nix are not just minor issues. These are at the core of the steep learning curve of Nix. These are also the reason why with Nix you are often 'dead in the water' and don't know how to fix a problem, and end up googling for hours or asking somebody else who knows more about Nix.
And these issues cannot be fixed by having "better docs" or a better tutorial.
If somebody were to rewrite the language today then my suggestion would be to make it statically typed, like Haskell. I would probably also make it strict, with optional laziness, although that's less important imo. And providing good error messages and good debuggability should be explicit design goals.
Making it statically typed would necessitate a different overall design, since many of the dynamic tricks would not be possible any more, but I suspect that you would end up with a better overall design in the end.
I think you would just make it Haskell. I don’t know the exact chronology of Nix (the language), so I can’t comment with full confidence, but Real World Haskell (written by my utterly brilliant former colleague Bryan O’Sullivan /humblebrag) was published in 2008, and the Haskell described there has monad transformers and all kinds of nifty stuff. I think Eelco’s thesis was published around the same time? It’s possible that my dates are off and they really needed to rig up their own System F machine, but I suspect it just seemed cool and now we’re stuck with it.
Nix (the language) stomps like JSON or YAML for writing config files, it’s actually not bad at that, but it’s a disaster in its contemporary role.
Oh thank God, I thought I was the only one. A while back, I posted "a lazy dynamically-typed language (in my book) perfectly combines the run-time reasonability of Haskell with all the development-time safety of LISP" on the Discourse, and the reply "I love the lazy functional minimal expression-oriented JSON-like packaging DSL that is nix" got 13 Likes. I thought I was going completely insane.
You’re not crazy. Duck-typing a lazy functional language in a zillion KLOC repo with broken stack traces is crazy.
At my work those “O RLY” meme book covers are in vogue this week, and my last one was “Nix Haskell Overlays for Fun and Murder: Homicidal Ideation for the Working Hacker”.
It was my job to get Clash (the VHDL/Verilog synthesis tool on Haskell) building under Bazel via Tweag’s rules_haskell and playing nice with all our other Haskell shit. That’s 14 hours straight I’m never getting back.
But… because it’s Nix I never have to do that again. Even if I upgrade everything else I can keep nixpkgs at that version in my flake.lock and that will always work forever. So I’m cool with it.
> But… because it’s Nix I never have to do that again.
This is the killer feature for me. My use-case for Nix is vastly different from yours, as I use it for centralized declarative configuration through NixOS, but I have an immense sense of satisfaction knowing that hell can freeze over but I'm still only a single command away from my exact setup running on (almost) any machine on the planet.
This might not matter much for most people, but making my machine work for me is something that took me years, and I can sleep well knowing that it's finally safe under Nix vs. multiple different configuration files for multiple different programs scattered across my filesystem.
Our use cases might not be that different. Whether it’s Mac laptops, physical dev machines, or cloud instances, our domain/scale is such that our machines are “pets” rather than “cattle”, i.e. they have names.
Making NVIDIA drivers work with either Xorg on someone’s desk or CuDNN on the GPU boxes with like two friggin lines in the NixOS stuff is like, doing drugs or something in terms of sheer feeling great after years of chasing that shit all around Ubuntu’s broken-ass standard of living.
> These alternatives are far more familiar and easier to work with. You can get most (but not all) of the benefits of NixOS with far less work.
> Worse is better.
There are some potential alternatives out there that seem inspired by Nix but which are less strict. I think a system that offers and manages Nix's purity but doesn't require it could end up the final winner some day. Shea Levy had some ideas for eventually achieving this in Nix that he called 'Nomia' (probably riffing on the name of Sander van der Burg's Dysnomia, I'd guess), but his life circumstances changed before the project really materialized. But Nix still could be the place where that is first achieved.
Meanwhile new container image-building DSLs like Dagger¹ and Bass² look a lot like a Nix which is impure by default, and bash builders are replaced with BuildKit directives. And there's Denxi³, which looks more like a Nix that lets you reduce its purity when you want to.
For now, though, the Nix ecosystem is still vastly more capable than these alternatives because of all the incredible work that Nixpkgs represents. Nix we know it today could still end up the concrete basis for what succeeds it, and not just an inspiration.
My experience was seriously different.
I started with Slackware, switched to Arch and after a failed attempt to run some AUR scripts, I was tired and switched to NixOS.
Incidentally, I treat both my Emacs config and my NixOS and home-manager configurations as Igors. They are never really finished. In this sense, the "many months of effort to get to that point" does not matter.
Also, with NixOS I feel I can be bolder than with other distros, knowing with way more accuracy what is breaking my configuration.
> Yes, Docker is an ugly duck compared to Nix, but it brings 90% of the reproducibility benefits and probably everyone on your team can be up and running in hours.
I’ve religiously updated a new-linux-install.sh. The majority of packages I build from source, and the remaining I use the distro’s package manager. I’m well aware that nix is a very different beast, but in practice this gives me basically the same outcome. Nix is incredibly interesting to me, but all the painful stories have kept me on my very simple and long bash script.
The primary issue for me has been that Nix is a very deep rabbit hole. You can spend enormous amounts of time on making your configuration more functional and declarative. Pretty much like you can spend enormous amounts of time on customizing an Emacs configuration. It's hard to strike a balance. And outside declaratively defined infrastructure (servers), it's probably not really worth it. I could almost fully reproduce my NixOS system (there is always some mutable state left) with a single command. But takes many months of effort to get to that point. On the other hand, I can set up a fresh macOS or Fedora Silverblue systems with all my customizations in 1 or 2 hours and have to do that maybe once of twice a year? So, ¯\_(ツ)_/¯.
I think the balance is different when you manage a lot of servers and most servers can be defined as a function with a small number of varying parameters.
The other part of the rabbit hole is that software breaks frequently in Nix. Upstreams do not really develop things under the assumption of a non-FHS, immutable system. So, you were going to work on something, but before you know it is 30 minutes later because you ended up fixing some package you need and that broke. Similarly, you'll end up packaging a lot of stuff and spending quite some time making it fit the Nix mold (looking at you Python packages that mutate in-place, sigh).
I love Nix as a principle -- it's declarative, immutable, pure, reproducible. But in practice, you can reap many of the same benefits from impure, inferior alternatives, with far less work. Yes, Docker is an ugly duck compared to Nix, but it brings 90% of the reproducibility benefits and probably everyone on your team can be up and running in hours. Rust's cargo doesn't allow you to specify every non-Rust dependency exactly, but with a Cargo.lock file you can make most of your build reproducible. NixOS is a clean, immutable, declarative system, but other systems offer a subset of its features, such as atomic upgrades/rollbacks, immutable root, and isolated applications (e.g. Fedora SilverBlue + Flatpaks, Fedora IoT, macOS). These alternatives are far more familiar and easier to work with. You can get most (but not all) of the benefits of NixOS with far less work.
Worse is better.