From ‘guix environment’ to ‘guix shell’
There are times when what looked like the right design choice some years
back comes out as an odd choice as time passes. The beloved guix environment
tool is having that fate. Its command-line interface has become
non-intuitive and annoying for the most common use cases. Since it
could not be changed without breaking compatibility in fundamental ways,
we devised a new command meant to progressively replace it; guix shell
—that’s
the name we unimaginatively ended up with—has just landed after a
three-week review period, itself a
followup to discussions and hesitations on the best course of action.
This post introduces guix shell
, how it differs from guix environment
, the choices we made, and why we hope you will like it.
The story of guix environment
The guix environment
command started its life in
2014,
when Guix was a two-year old baby and the whole community could fit in a
small room. It had one purpose: “to assist hackers in creating
reproducible development environments”. It was meant to be similar in
spirit to VirtualEnv or
Bundler, but universal—not limited to a single
language. You would run:
guix environment inkscape
… and obtain an interactive shell with all the packages needed to hack
on Inkscape; in that shell, the relevant environment variables—PATH
,
CPATH
, PKG_CONFIG_PATH
, and so on—would automatically point to a
profile created on the fly and containing the compiler, libraries, and
tools Inkscape depends on, but not Inkscape itself.
Only a year later did it become clear that there are cases where one
would want to create an environment containing specific packages, rather
than an environment containing the dependencies of packages. To
address that, David Thompson proposed the --ad-hoc
option:
guix environment --ad-hoc inkscape -- inkscape
… would create an environment containing only Inkscape, and would then
launch the inkscape
command in that environment. Many features were
added over the years, such as the invaluable --container
option,
but these two modes, development and “ad hoc”, are the guts of it.
Fast forward six years: today, there’s consensus that the name
--ad-hoc
is confusing for newcomers and above all, that the “ad hoc”
mode should be the default. This is the main problem that guix shell
addresses.
Doing what you’d expect
Changing the default mode from “development environment” to “ad hoc” is technically easy, but how to do that without breaking compatibility is harder. This led to lengthy discussions, including proposals of mechanisms to choose between the new and old semantics.
In the end, keeping the guix environment
name while allowing it to
have different semantics was deemed dangerous. For one thing, there’s
lots of material out there that demoes guix environment
—blog posts,
magazine articles, on-line courses—and it would have been impossible to
determine whether they refer to the “new” or to the “old” semantics. We
reached the conclusion that it
would be easier to use a new command name and to eventually deprecate
guix environment
.
With guix shell
, the default is to create an environment that contains
the packages that appear on the command line; to launch Inkscape, run:
guix shell inkscape -- inkscape
The --ad-hoc
option is gone! Likewise, to spawn an ephemeral
development environment containing Python and a couple of libraries,
run:
guix shell python python-numpy python-scipy -- python3
Now, if you want, say, the development environment of Inkscape, add the
--development
or -D
option right before:
guix shell -D inkscape
You can add Git and GDB on top of it like so:
guix shell -D inkscape git gdb
(Note that -D
only applies to the immediately following package,
inkscape
in this case.) It’s more concise and more natural than with
guix environment
. As can be seen in the
manual,
all the other options supported by guix environment
remain available
in guix shell
.
Short-hands for development environments
A convention that’s become quite common is for developers to provide a
guix.scm
at the top of their project source tree, so that others can
start a development environment right away:
guix environment -l guix.scm
The guix.scm
file would contain a package
definition
for the project at hand, as in this
example.
This option is known as -f
in guix shell
, for consistency with other
commands, and the equivalent command is:
guix shell -D -f guix.scm
Since all Guix commands accept a
“manifest”
with -m
, another option is to provide a manifest.scm
file and to
run:
guix shell -m manifest.scm
“Wouldn’t it be nice if guix shell
would automatically follow these
conventions when not given any argument?”, some
suggested.
As in the case of Bundler, direnv, or typical
build tools from Meson to Make, having a default file name can save
typing and contribute to a good user experience for frequently-used
commands. In this spirit, guix shell
automatically loads guix.scm
or manifest.scm
, from the current directory or an ancestor thereof,
such that entering a project to hack on it is as simple as:
cd ~/my/project/src
guix shell
Worry not: guix shell
loads guix.scm
or manifest.scm
if and only
if you have first added its directory to
~/.config/guix/shell-authorized-directories
. Otherwise guix shell
warns you and prints a hint that you can copy/paste if you want to
authorize the directory.
Caching environments
With that in place, guix shell
can pretty much fill the same role as
direnv and similar tools, with one difference though: speed. When all
the packages are already in store, guix shell
can take one to a few
seconds to run, depending on the package set, on whether you’re using a
solid state device (SSD) or a “spinning” hard disk, and so on. It’s
acceptable but prohibitively slow for direnv-like use cases.
To address that, guix shell
maintains a profile cache for the -D -f guix.scm
and -m manifest.scm
cases. On a hot cache, it runs in
0.1 second. All it has to do is fork a shell with the right environment
variable definitions; it does not talk to guix-daemon
, and it does not
even read guix.scm
or manifest.scm
(it’s possible to forcefully
update the cache with --rebuild-cache
).
That makes guix shell
usable even for short-lived commands like
make
:
guix shell -- make
Hopefully it’ll change the way we use the tool!
The shell doctor
While revamping this command-line interface, the idea of a “shell
doctor” came up. In interactive use, guix shell
sets environment
variables and spawns a shell, but it’s not uncommon for the shell to
mess up with the whole environment. Why? Because, contrary to
documented
practice,
it’s quite common for users to define or override environment variables
in the startup files of non-login shells, ~/.bashrc
for Bash,
~/.zshrc
for Zsh. Instead, environment variable definitions should go
to the startup file of login shells—~/.bash_profile
, ~/.profile
,
or similar. But let’s face it: it’s a subtle distinction that few of us
know or care about.
As a result, users of Guix, especially on distros other than Guix
System, would often be disappointed when running guix environment --pure
and yet find that PATH
contains non-Guix entries, that
there’s a bogus LD_LIBRARY_PATH
definition, and whatnot. Now, they
can call the doctor, so to speak, to obtain a diagnosis of the health of
their shell by adding the --check
flag:
guix shell --check python python-numpy
The command creates an environment containing Python and NumPy, spawns
an interactive shell, checks the environment variables as seen by the
shell, and prints a warning if PATH
or PYTHONPATH
in this case have
been overridden. It does not tell users where the problem comes from—it
cannot guess—but it tells them if something’s wrong, which is a first
step.
Of course, the best way to sidestep these problems is to pass
--container
, which gives a fresh, isolated environment that does not
contain those startup files. That’s not always an option though, for
instance on systems lacking support for unprivileged user namespaces, so
--check
comes in handy there.
Try it!
Just run guix pull
to get this shiny new guix shell
thingie!
If you don’t feel ready yet, that’s OK: guix environment
won’t
disappear overnight. We have a written
commitment
to keep it around until May, 1st 2023. Though overall, we hope you’ll
find the guix shell
interface easier to use and compelling enough that
you’ll be willing to switch overnight!
About GNU Guix
GNU Guix is a transactional package manager and an advanced distribution of the GNU system that respects user freedom. Guix can be used on top of any system running the Hurd or the Linux kernel, or it can be used as a standalone operating system distribution for i686, x86_64, ARMv7, AArch64 and POWER9 machines.
In addition to standard package management features, Guix supports transactional upgrades and roll-backs, unprivileged package management, per-user profiles, and garbage collection. When used as a standalone GNU/Linux distribution, Guix offers a declarative, stateless approach to operating system configuration management. Guix is highly customizable and hackable through Guile programming interfaces and extensions to the Scheme language.
Unless otherwise stated, blog posts on this site are copyrighted by their respective authors and published under the terms of the CC-BY-SA 4.0 license and those of the GNU Free Documentation License (version 1.3 or later, with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts).