From ‘guix environment’ to ‘guix shell’

Ludovic Courtès — 26. oktober, 2021

There are times when what looked like the right design choice some years back comes out as an odd choice as time passes. The beloved guix environment tool is having that fate. Its command-line interface has become non-intuitive and annoying for the most common use cases. Since it could not be changed without breaking compatibility in fundamental ways, we devised a new command meant to progressively replace it; guix shell—that’s the name we unimaginatively ended up with—has just landed after a three-week review period, itself a followup to discussions and hesitations on the best course of action.

This post introduces guix shell, how it differs from guix environment, the choices we made, and why we hope you will like it.

The story of guix environment

The guix environment command started its life in 2014, when Guix was a two-year old baby and the whole community could fit in a small room. It had one purpose: “to assist hackers in creating reproducible development environments”. It was meant to be similar in spirit to VirtualEnv or Bundler, but universal—not limited to a single language. You would run:

guix environment inkscape

… and obtain an interactive shell with all the packages needed to hack on Inkscape; in that shell, the relevant environment variables—PATH, CPATH, PKG_CONFIG_PATH, and so on—would automatically point to a profile created on the fly and containing the compiler, libraries, and tools Inkscape depends on, but not Inkscape itself.

Only a year later did it become clear that there are cases where one would want to create an environment containing specific packages, rather than an environment containing the dependencies of packages. To address that, David Thompson proposed the --ad-hoc option:

guix environment --ad-hoc inkscape -- inkscape

… would create an environment containing only Inkscape, and would then launch the inkscape command in that environment. Many features were added over the years, such as the invaluable --container option, but these two modes, development and “ad hoc”, are the guts of it.

Fast forward six years: today, there’s consensus that the name --ad-hoc is confusing for newcomers and above all, that the “ad hoc” mode should be the default. This is the main problem that guix shell addresses.

Doing what you’d expect

Changing the default mode from “development environment” to “ad hoc” is technically easy, but how to do that without breaking compatibility is harder. This led to lengthy discussions, including proposals of mechanisms to choose between the new and old semantics.

In the end, keeping the guix environment name while allowing it to have different semantics was deemed dangerous. For one thing, there’s lots of material out there that demoes guix environment—blog posts, magazine articles, on-line courses—and it would have been impossible to determine whether they refer to the “new” or to the “old” semantics. We reached the conclusion that it would be easier to use a new command name and to eventually deprecate guix environment.

With guix shell, the default is to create an environment that contains the packages that appear on the command line; to launch Inkscape, run:

guix shell inkscape -- inkscape

The --ad-hoc option is gone! Likewise, to spawn an ephemeral development environment containing Python and a couple of libraries, run:

guix shell python python-numpy python-scipy -- python3

Now, if you want, say, the development environment of Inkscape, add the --development or -D option right before:

guix shell -D inkscape

You can add Git and GDB on top of it like so:

guix shell -D inkscape git gdb

(Note that -D only applies to the immediately following package, inkscape in this case.) It’s more concise and more natural than with guix environment. As can be seen in the manual, all the other options supported by guix environment remain available in guix shell.

Short-hands for development environments

A convention that’s become quite common is for developers to provide a guix.scm at the top of their project source tree, so that others can start a development environment right away:

guix environment -l guix.scm

The guix.scm file would contain a package definition for the project at hand, as in this example. This option is known as -f in guix shell, for consistency with other commands, and the equivalent command is:

guix shell -D -f guix.scm

Since all Guix commands accept a “manifest” with -m, another option is to provide a manifest.scm file and to run:

guix shell -m manifest.scm

“Wouldn’t it be nice if guix shell would automatically follow these conventions when not given any argument?”, some suggested. As in the case of Bundler, direnv, or typical build tools from Meson to Make, having a default file name can save typing and contribute to a good user experience for frequently-used commands. In this spirit, guix shell automatically loads guix.scm or manifest.scm, from the current directory or an ancestor thereof, such that entering a project to hack on it is as simple as:

cd ~/my/project/src
guix shell

Worry not: guix shell loads guix.scm or manifest.scm if and only if you have first added its directory to ~/.config/guix/shell-authorized-directories. Otherwise guix shell warns you and prints a hint that you can copy/paste if you want to authorize the directory.

Caching environments

With that in place, guix shell can pretty much fill the same role as direnv and similar tools, with one difference though: speed. When all the packages are already in store, guix shell can take one to a few seconds to run, depending on the package set, on whether you’re using a solid state device (SSD) or a “spinning” hard disk, and so on. It’s acceptable but prohibitively slow for direnv-like use cases.

To address that, guix shell maintains a profile cache for the -D -f guix.scm and -m manifest.scm cases. On a hot cache, it runs in 0.1 second. All it has to do is fork a shell with the right environment variable definitions; it does not talk to guix-daemon, and it does not even read guix.scm or manifest.scm (it’s possible to forcefully update the cache with --rebuild-cache).

That makes guix shell usable even for short-lived commands like make:

guix shell -- make

Hopefully it’ll change the way we use the tool!

The shell doctor

While revamping this command-line interface, the idea of a “shell doctor” came up. In interactive use, guix shell sets environment variables and spawns a shell, but it’s not uncommon for the shell to mess up with the whole environment. Why? Because, contrary to documented practice, it’s quite common for users to define or override environment variables in the startup files of non-login shells, ~/.bashrc for Bash, ~/.zshrc for Zsh. Instead, environment variable definitions should go to the startup file of login shells—~/.bash_profile, ~/.profile, or similar. But let’s face it: it’s a subtle distinction that few of us know or care about.

As a result, users of Guix, especially on distros other than Guix System, would often be disappointed when running guix environment --pure and yet find that PATH contains non-Guix entries, that there’s a bogus LD_LIBRARY_PATH definition, and whatnot. Now, they can call the doctor, so to speak, to obtain a diagnosis of the health of their shell by adding the --check flag:

guix shell --check python python-numpy

The command creates an environment containing Python and NumPy, spawns an interactive shell, checks the environment variables as seen by the shell, and prints a warning if PATH or PYTHONPATH in this case have been overridden. It does not tell users where the problem comes from—it cannot guess—but it tells them if something’s wrong, which is a first step.

Of course, the best way to sidestep these problems is to pass --container, which gives a fresh, isolated environment that does not contain those startup files. That’s not always an option though, for instance on systems lacking support for unprivileged user namespaces, so --check comes in handy there.

Try it!

Just run guix pull to get this shiny new guix shell thingie!

If you don’t feel ready yet, that’s OK: guix environment won’t disappear overnight. We have a written commitment to keep it around until May, 1st 2023. Though overall, we hope you’ll find the guix shell interface easier to use and compelling enough that you’ll be willing to switch overnight!

About GNU Guix

GNU Guix is a transactional package manager and an advanced distribution of the GNU system that respects user freedom. Guix can be used on top of any system running the Hurd or the Linux kernel, or it can be used as a standalone operating system distribution for i686, x86_64, ARMv7, AArch64 and POWER9 machines.

In addition to standard package management features, Guix supports transactional upgrades and roll-backs, unprivileged package management, per-user profiles, and garbage collection. When used as a standalone GNU/Linux distribution, Guix offers a declarative, stateless approach to operating system configuration management. Guix is highly customizable and hackable through Guile programming interfaces and extensions to the Scheme language.