Creating bundles with guix pack
Guix just got a new command,
dubbed guix pack
,
which we think many developers will find useful.
Last week we were celebrating the release of GNU Guile 2.2.0, the Scheme implementation that powers Guix. This is a major milestone and Guile developers naturally wanted to make it easy for users to discover all the goodies of 2.2.0 as soon as possible. One of the major roadblocks to that, as for any non-trivial piece of software, is deployment: because your distro is unlikely to have Guile 2.2.0 packaged on Day 1, you have to build it by yourself, which means getting the right dependencies installed and then building Guile itself. That’s not difficult for a developer, but it’s certainly cumbersome.
Andy Wingo, the driving force behind Guile, thought that it would be
nice to propose a binary tarball of Guile 2.2.0 on the day of its
release. Guix had already been providing
binary tarballs
for a couple of years, so why not do the same for Guile? Essentially,
the new guix pack
command is a generalization of what Guix was already
using.
Making packs
So how does it work? The basic idea is simple: you type
guix pack guile
and the command returns in /gnu/store
a good old tarball that
contains binaries for Guile and all its dependencies. If you run, say,
guix pack guile emacs geiser
then you get a complete “Guile SDK” containing Guile, Emacs, Geiser, and all their dependencies.
When you extract the tarball, you get a /gnu/store
directory with a
bunch of sub-directories with
these long hashes,
one of which is the “profile” containing Guile, Emacs, and Geiser.
You wouldn’t want to ask users to type
/gnu/store/war325pv1iixj13k6y8yplzagpknfn0c-profile/bin/guile
to
launch Guile, though. So guix pack
has a command-line option to
create symlinks in the image.
guix pack -S /opt/gnu/bin=bin guile emacs geiser
The command above creates a /opt/gnu/bin
symlink to the bin
directory of the profile in the tarball, such that users can simply type
/opt/gnu/bin/guile
to run Guile.
Recipients of a binary tarball are expected to either extract it in
their root file system (yes!) where it will create /gnu
and /opt/gnu
in this case:
# cd /
# tar xf /path/to/pack.tar.gz
# /opt/gnu/bin/guile --version
guile (GNU Guile) 2.2.0
… or they can chroot into it, possibly relying on user namespaces and thereby avoiding root privileges:
$ mkdir /tmp/pack
$ cd /tmp/pack
$ tar xf /path/to/pack.tar.gz
$ unshare -mrf chroot . /opt/gnu/bin/guile --version
guile (GNU Guile) 2.2.0
The good thing with this is that, because Guix captures the complete dependency graph of packages, the tarball contains everything that’s needed to run Guile and is going to work in exactly the same way on any system that runs the kernel Linux!
Bells and whistles
Of course a popular approach to run such “application bundles” is
Docker. Since the image format for Docker is
documented
and fairly easy to produce, we added an option to produce images in this
format (Ricardo Wurmus
initially contributed Docker support
for the low-level
guix archive
tool but we found that it made more sense to have it in guix pack
):
guix pack -f docker -S /opt/gnu=/ guile emacs geiser
The resulting tarball can be passed to
docker load
,
and people can then use docker run
to actually run the application.
One of the goodies that comes for free is cross-compilation: Guix supports cross-compilation, so you can create a pack consisting of software cross-compiled for a given platform, specified by the usual GNU triplet. For example, the following command creates a pack with binaries for GNU/Linux on ARMv7:
guix pack --target=arm-linux-gnueabihf guile
… while the command below creates a pack with Windows binaries using the MinGW cross-compiler:
guix pack --target=i686-w64-mingw32 guile
All the
package transformation options
that Guix supports are available to guix pack
. Let’s say you’re a
developer of a large piece of software such as a web browser like
IceCat and you’d like your users to test
whether the current master branch actually fixes the bug you attempted
to fix. In this case, you can build a pack of IceCat, but replace the
source that’s specified in the distribution with the snapshot of master
you’re interested in:
guix pack icecat --with-source=./icecat-48.8.0.master.tar.gz
Of course the resulting pack is going to be pretty big in this case, but I’m sure the general pattern can be useful.
Wait, didn’t you say that “app bundles get it wrong”?
It turns out that we Guix developers have been saying that binary “application bundles” à la Docker are problematic for a number of reasons:
- Composability: each bundle comes with a complete operating system, minus the kernel, and there is little or no sharing happening among bundles, notably in terms of disk space and memory usage.
- Security updates: since an “app bundle” is essentially a complete operating system, one has to be careful and apply security updates to all the software in each bundle. Unfortunately, that doesn’t always happen as has been famously reported on several occasions.
- Reproducibility: Docker images, for instance, are often hardly
“reproducible” in the sense of a
reproducible build process.
First,
Dockerfile
s start out with a “base layer” that is typically a huge binary blob of some major distro. On top of that, they run a number of commands such asapt-get install
whose result likely depends on the time at which they are run. Docker’s best practices document suggests ways to mitigate the problem, such as “version pinning”, but the whole approach remains rather brittle. - Experimentation: Once you have this big binary blob, sure you can run the application you wanted, but you can do little more than that—you may or may not be able to find the corresponding source code, and you’d have a hard time fiddling with one of the components of the software stack.
We pride ourselves with having a tool set that caters to some of the use cases that “app bundles” and “containerization” try to address while having none of these drawbacks. So how do Guix packs fit into that picture?
First of all, the intended use case is different: we view guix pack
as
a tool that makes it easy to try out a piece of software on a non-Guix
machine. But it is clear that for production, our recommendation is to
use Guix directly, to get security updates and generally address all the
above issues. :-)
That said, let’s see how these issues affect Guix packs. First,
composability of Guix packs turns out to be pretty good. If you receive
two different Guix packs for different pieces of software, you can
unpack both in your root directory (or union-mount them in the same
place): packages that differ have a different /gnu/store
file name
with a different hash, so they won’t collide; packages that are
identical (say the C library or GTK+) will have the same /gnu/store
file name so they’ll actually be shared.
That means that for security updates, you could always fetch a new pack of your application with the security updates and extract it in place. However, that requires you as a user to manually pay attention to vulnerabilities in all the software that comes with the pack, so clearly you’re better off using Guix instead and regularly upgrading. No wonders.
Packs themselves are reproducible bit-by-bit. If you know the Guix
commit that was used to build a given pack, you can thus run the same
guix pack
command on another machine and verify that you get the exact
same tarball. Currently not 100% of the packages Guix provides are
reproducible bit-by-bit; we’re getting closer to that goal though, in
part due to the fact that Guix builds are isolated by default, and also
thanks to the efforts of everyone in the
Reproducible Builds project to
address sources of non-determinism in free software.
Because Guix packs are reproducible, you can not only reproduce the exact same pack but also create packs with variants of the software—for instance, changing the version of one of the packages in the stack. Of course this part requires you to have Guix installed somewhere, but at least you can easily fiddle with the software stack and “compile” your own variant of the software stack down to a new pack.
We hope you’ll enjoy packs and Guix, and would welcome your feedback on
the guix-devel
mailing list
and on #guix
on Freenode!
About GNU Guix
GNU Guix is a transactional package manager for the GNU system. The Guix System Distribution or GuixSD is an advanced distribution of the GNU system that relies on GNU Guix and respects the user's freedom.
In addition to standard package management features, Guix supports transactional upgrades and roll-backs, unprivileged package management, per-user profiles, and garbage collection. Guix uses low-level mechanisms from the Nix package manager, except that packages are defined as native Guile modules, using extensions to the Scheme language. GuixSD offers a declarative approach to operating system configuration management, and is highly customizable and hackable.
GuixSD can be used on an i686 or x86_64 machine. It is also possible to use Guix on top of an already installed GNU/Linux system, including on mips64el and armv7.
Unless otherwise stated, blog posts on this site are copyrighted by their respective authors and published under the terms of the CC-BY-SA 4.0 license and those of the GNU Free Documentation License (version 1.3 or later, with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts).