Multi-dimensional transactions and rollbacks, oh my!
One of the highlights of version
0.15.0
was the overhaul of guix pull
,
the command that updates Guix and its package collection. In Debian
terms, you can think of guix pull
as:
apt-get update && apt-get install apt
Let’s be frank, guix pull
does not yet run as quickly as this
apt-get
command—in the “best case”, when pre-built binaries are
available, it currently runs in about 1m30s on a recent laptop. More
about the performance story in a future post…
One of the key features of the new guix pull
is the ability to roll
back to previous versions of Guix. That’s a distinguishing feature
that opens up new possibilities.
“Profile generations”
Transactional upgrades and rollbacks have been a distinguishing feature of Guix since Day 1. They come for free as a consequence of the functional package management model inherited from the Nix package manager. To many users, this alone is enough to justify using a functional package manager: if an upgrade goes wrong, you can always roll back. Let’s recap how this all works.
As a user, you install packages in your own profile, which defaults to
~/.guix-profile
. Then from time to time you update Guix and its
package collection:
$ guix pull
This updates ~/.config/guix/current
, giving you an updated guix
executable along with an updated set of packages. You can now upgrade
the packages that are in your profile:
$ guix package -u
The following packages will be upgraded:
diffoscope 93 → 96 /gnu/store/…-diffoscope-96
emacs 25.3 → 26.1 /gnu/store/…-emacs-26.1
gimp 2.8.22 → 2.10.4 /gnu/store/…-gimp-2.10.4
gnupg 2.2.7 → 2.2.9 /gnu/store/…-gnupg-2.2.9
The upgrade creates a new generation of your profile—the previous generation of your profile, with diffoscope 93, emacs 25.3, and so on is still around. You can list profile generations:
$ guix package --list-generations
Generation 1 Jun 08 2018 20:06:21
diffoscope 93 out /gnu/store/…-diffoscope-93
emacs 25.3 out /gnu/store/…-emacs-25.3
gimp 2.8.22 out /gnu/store/…-gimp-2.8.22
gnupg 2.2.7 out /gnu/store/…-gnupg-2.2.7
python 3.6.5 out /gnu/store/…-python-3.6.5
Generation 2 Jul 12 2018 12:42:08 (current)
- diffoscope 93 out /gnu/store/…-diffoscope-93
- emacs 25.3 out /gnu/store/…-emacs-25.3
- gimp 2.8.22 out /gnu/store/…-gimp-2.8.22
- gnupg 2.2.7 out /gnu/store/…-gnupg-2.2.7
+ diffoscope 96 out /gnu/store/…-diffoscope-96
+ emacs 26.1 out /gnu/store/…-emacs-26.1
+ gimp 2.10.4 out /gnu/store/…-gimp-2.10.4
+ gnupg 2.2.9 out /gnu/store/…-gnupg-2.2.9
That shows our two generations with the diff between Generation 1 and
Generation 2. We can at any time run guix package --roll-back
and get
our previous versions of gimp, emacs, and so on. Each generation is
just a bunch of symlinks to those packages, so what we have looks like
this:
Notice that python was not updated, so it’s shared between both generations. And of course, all the dependencies that didn’t change in between—e.g., the C library—are shared among all packages.
guix pull
generations
Like I wrote above, guix pull
brings the latest set of package
definitions from Git master
. The Guix package collection usually
contains only the latest version of each package; for example, current
master
only has version 26.1 of Emacs and version 2.10.4 of the GIMP
(there are notable exceptions such as GCC or Python.) Thus, guix package -i gimp
, from today’s master, can only install gimp 2.10.4.
Often, that’s not a problem: you can keep old profile generations
around, so if you really need that older version of Emacs, you can run
it from your previous generation.
Still, having guix pull
keep track of the changes to Guix and its
package collection is useful. Starting from 0.15.0, guix pull
creates
a new generation, just like guix package
does. After you’ve run guix pull
, you can now list Guix generations as well:
$ guix pull -l
Generation 10 Jul 14 2018 00:02:03
guix 27f7cbc
repository URL: https://git.savannah.gnu.org/git/guix.git
branch: origin/master
commit: 27f7cbc91d1963118e44b14d04fcc669c9618176
Generation 11 Jul 20 2018 10:44:46
guix 82549f2
repository URL: https://git.savannah.gnu.org/git/guix.git
branch: origin/master
commit: 82549f2328c59525584b92565846217c288d8e85
14 new packages: bsdiff, electron-cash, emacs-adoc-mode,
emacs-markup-faces, emacs-rust-mode, inchi, luakit, monero-gui,
nethack, openbabel, qhull, r-txtplot, stb-image, stb-image-write
52 packages upgraded: angband@4.1.2, aspell-dict-en@2018.04.16-0,
assimp@4.1.0, bitcoin-core@0.16.1, botan@2.7.0, busybox@1.29.1,
…
Generation 12 Jul 23 2018 15:22:52 (current)
guix fef7bab
repository URL: https://git.savannah.gnu.org/git/guix.git
branch: origin/master
commit: fef7baba786a96b7a3100c9c7adf8b45782ced37
20 new packages: ccrypt, demlo, emacs-dired-du,
emacs-helm-org-contacts, emacs-ztree, ffmpegthumbnailer,
go-github-com-aarzilli-golua, go-github-com-kr-text,
go-github-com-mattn-go-colorable, go-github-com-mattn-go-isatty,
go-github-com-mgutz-ansi, go-github-com-michiwend-golang-pretty,
go-github-com-michiwend-gomusicbrainz, go-github-com-stevedonovan-luar,
go-github-com-wtolson-go-taglib, go-github-com-yookoala-realpath,
go-gitlab-com-ambrevar-damerau, go-gitlab-com-ambrevar-golua-unicode,
guile-pfds, u-boot-cubietruck
27 packages upgraded: c-toxcore@0.2.4, calibre@3.28.0,
emacs-evil-collection@20180721-2.5d739f5,
…
The nice thing here is that guix pull
provides high-level information
about the differences between two subsequent generations of Guix.
In the end, Generation 1 of our profile was presumably built with Guix
Generation 11, while Generation 2 of our profile was built with Guix
Generation 12. We have a clear mapping between Guix generations as
created by guix pull
and profile generations as created with guix package
:
Each generation created by guix pull
corresponds to one commit in the
Guix repo. Thus, if I go to another machine and run:
$ guix pull --commit=fef7bab
then I know that I get the exact same Guix instance as my Generation 12 above. From there I can install diffoscope, emacs, etc. and I know I’ll get the exact same binaries as those I have above, thanks to reproducible builds.
These are very strong guarantees in terms of reproducibility and provenance tracking—properties that are typically missing from “applications bundles” à la Docker.
In addition, you can easily run an older Guix. For instance, this is how you would install the version of gimp that was current as of Generation 10:
$ ~/.config/guix/current-10-link/bin/guix package -i gimp
At this point your profile contains gimp coming from an old Guix along with packages installed from the latest Guix. Past and present coexist in the same profile. The historical dimension of the profile no longer matches exactly the history of Guix itself.
Composing Guix revisions
Some people have expressed interest in being able to compose packages coming from different revisions of Guix—say to create a profile containing old versions of Python and NumPy, but also the latest and greatest GCC. It may seem far-fetched but it has very real applications: there are large collections of scientific packages and in particular bioinformatics packages that don’t move as fast as our beloved flagship free software packages, and users may require ancient versions of some of the tools.
We could keep old versions of many packages but maintainability costs
would grow exponentially. Instead, Guix users can take advantage of the
version control history of Guix itself to mix and match packages coming
from different revisions of Guix. As shown above, it’s already possible
to achieve this by running the guix
program off the generation of
interest. It does the job, but can we do better?
In the process of enhancing guix pull
we developed a high-level API
that allows an instance of Guix to “talk” to a different instance of
Guix—an
“inferior”. It’s
what allows guix pull
to display the list of packages that were added
or upgraded between two revisions. The next logical step will be to
provide seamless integration of packages coming from an inferior. That
way, users would be able to refer to “past” package graphs right from a
profile
manifest
or from the command-line. Future work!
On coupling
The time traveler in you might be wondering: Why are package definitions coupled with the package manager, doesn’t it make it harder to compose packages coming from different revisions? Good point!
Tight coupling certainly complicates this kind of composition: we can’t
just have any revision of Guix load package definitions from any other
revision; this could fail altogether, or it could provide a different
build result. Another potential issue is that guix pull
ing an older
revision not only gives you an older set of packages, it also gives you
older tools, bug-for-bug.
The reason for this coupling is that a package definition like this
one
doesn’t exist in a vacuum. Its meaning is defined by the implementation
of package
objects,
by
gnu-build-system
,
by a number of lower-level
abstractions that are all
defined as extensions of the Scheme language in Guix itself, and
ultimately by Guile, which
implements the language Guix is written in. Each instance created by
guix pull
brings all these components. Because Guix is implemented as
a set of programming language extensions and libraries, that package
definitions depend on all these parts becomes manifest. Instead of
being frozen, the APIs and package definitions evolve together, which
gives us developers a lot of freedom on the changes we can make.
Nix results from a different design choice. Nix-the-package-manager implements the Nix language, which acts as a “frozen” interface. Package definitions in Nixpkgs are written in that language, and a given version of Nix can possibly interpret both current and past package definitions without further ado. The Nix language does evolve though, so at one point an old Nix inevitably becomes unable to evaluate a new Nixpkgs, and vice versa.
These two approaches make different tradeoffs. Nix’ loose coupling simplifies the implementation and makes it easy to compose old and new package definitions, to some extent; Guix’ tight coupling makes such composition more difficult to implement, but it leaves developers more freedom and, we hope, may support “time travels” over longer period of times. Time will tell!
It’s like driving a DeLorean
Picture of a DeLorean cabin by Oto Godfrey and Justin Morton, under CC-BY-SA 4.0.
That profile generations are kept around already gave users a time
machine of sorts—you can always roll back to a previous state of your
software environment. With the addition of roll-back support for guix pull
, this adds another dimension to the time machine: you can
roll-back to a previous state of Guix itself and from there create
alternative futures or even mix bits from the past with bits from the
present. We hope you’ll enjoy it!
Unless otherwise stated, blog posts on this site are copyrighted by their respective authors and published under the terms of the CC-BY-SA 4.0 license and those of the GNU Free Documentation License (version 1.3 or later, with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts).