GMT now available in nixpkgs

Lounge
1

Hi Folks,

I’m guessing that the intersection between the set of all GMT users (or even people doing geo-something type work), and the set of all nixpkgs users, is probably infinitesimally small (perhaps even null!), but throwing this out there in case it is interesting/helpful to anybody. Who knows, maybe some curious cartographic soul will decide to try out nix/nixos.

Just a heads up, that GMT v6.1.0 was recently added to nixpkgs, the official nix package repo. Right now it’s only on the master branch, but over a period of days it’ll hopefully trickle down into the other stable/unstable branches. Tested on macos and nixos (although tbh I haven’t used it on macos in a while).

What is nix/nixpkgs?

Nix is a package manager that runs on multiple platforms, including macos (so this presents an alternative to brew for mac users wanting to run GMT). Nixpkgs is the official “channel” (i.e. package repo) for the nix the package manager, while NixOS is an entire operating system based on nix. Nix is also the name of a declarative, strictly functional, configuration mini-language, that nixpkgs is based off of. In the sort of “canonical” nix workflow/setup, users write configuration files in the nix language, that define/declare the state of the system (or user) environment, which nix then consumes to determine what to install. However, it is also possible to install packages imperatively, like how one would use a more typical package manager like aptitude or brew (e.g. apt-get install and brew install becomes nix-env --install).

Why use nix?

The following excerpt from the official page, is how people usually try to sell nix

Nix builds packages in isolation from each other. This ensures that they are reproducible and don’t have undeclared dependencies, so if a package works on one machine, it will also work on another .

BUT, my favorite feature of nix is actually

Nix makes it trivial to share development and build environments for your projects, regardless of what programming languages and tools you’re using.

In practice, this works analogously to Anaconda’s environment mechanism, although in my experience it’s a lot more effective (e.g. nix avoids duplicating package installs across envs, so they are much much smaller than conda envs). I’ve been working on some ocean modeling lately, so for me this means that it’s super easy to work on my models from different computers.

Anyways, I’m monologuing now. Hope this helps somebody!

3 Likes
2

So … what are the PRO/CONS when compared with brew ?
Thanks

3

Guess it goes without saying that these are all just my opinion. I’ll try to stick to the high-level benefits that would be visible to the end user (if I’m being honest, a lot of the “pros” are more academic/technical in nature, rather than user-facing benefits).

Pros

  • Runs on arbitrary unix operating systems.
  • Easy to share dev environments across machines, or with other users.
  • Run multiple package versions simultaneously.
  • No “cruft buildup” over time. Packages are built in a chroot jail, and installed to an isolated read-only folder, so when they are removed, they usually can’t leave anything behind (other than maybe dotfiles).
  • You can always roll the system back to a previous state. “Uninstalling” packages doesn’t actually remove them until you force a “garbage collection”.
  • If you use the nix language for system (or project) management, then the system state can be checked into a version control system. Combined with the commit hash for your nixpkgs checkout, the system state is more or less reproducible (for some packages, possibly bit-for-bit reproducible).

For academic work, I think that last point is pretty exciting, since it means that so long as you check in a shell.nix file into your project root (analogous to conda’s environment.yml files), then in theory anybody else can reproduce your experiment, down to the exact version for each and every package you used.

Cons

  • Many of the above points are probably more “academic” in nature, and are things that most users probably don’t care about.
  • Have to learn a new programming language if you want to take full advantage of nix.
  • Not as widespread as brew, so some software might not already be packaged out of the box (usually pretty easy to add a package, but requires you know the language first).
  • Have to periodically run the nix garbage collector to remove old package versions. If you don’t, the size of the system will balloon VERY quickly.
  • Creates a root directory for storing build outputs (when macos Catalina was rolled out, this wound up being a big problem).
  • Doesn’t follow typical system-management idioms. For example, NixOS does NOT follow the FHS.
  • Because of the above point, arbitrary precompiled binaries typically don’t work out-of-the-box with nix packages (usually you have to patch the binary with a tool like patchelf to fix the lib paths, or you have to build things from scratch).
4

And how does it deal with libraries (shared / symbolic link / …).
If I work on a remote cluster, does this shared dev environments works too in my personal directory (via conda for example?)

5

And how does it deal with libraries (shared / symbolic link / …).

In nix, EVERYTHING is symlinked into the environment. When a package is “installed”, the build outputs are placed in /nix/store/<hash>-<package-name>, and then symlinked into the “correct” place. This includes binaries AND libraries.

If I work on a remote cluster, does this shared dev environments works too in my personal directory (via conda for example?)

This is a caveat that I haven’t found a solution for, since most clusters won’t just have nix installed by default (although I think there are some bio groups out there that run nix instead of a more traditional “module” system).

In theory though, you can relocate /nix to wherever is convenient, including a user level dir (e.g. your home dir), so you could just build nix yourself and then run it like that. I haven’t tried this though.

One thing to keep in mind is that since nix builds everything in isolation, it’s going to ignore anything that’s already installed on the cluster, and you’ll lose the benefit of any compiler/lib optimizations your sysadmin may have taken the time to implement. Unless of course you override the nix package definitions and implement these things yourself. This also means that your user dir will start to get really big, since nix will install its own version of EVERYTHING.

1 Like