Reproducible Builds Summit, 4th edition

Julien Lepiller, Gábor Boskovits, Ludovic Courtès, Ricardo Wurmus — 21. december, 2018

As it has become tradition, a sizeable delegation of Guix developers attended this year's Reproducible Builds Summit in Paris a little over one week ago. In the Mozilla offices around 50 people representing dozens of free software projects (and also Microsoft) got ready to share ideas and work together on reproducibility problems for three days. As the agenda unfolded we would split up into small groups, each focusing on a different issue. This blog post is an attempt to share impressions from some of these group sessions.

Opam

Opam is the OCaml package manager, and a number of sessions were dedicated to addressing reproducibility problems that affect Opam, such as the lack of accounting for common system packages. One of the outcomes of the session is a script to catch common sources of non-determinism, which should improve the quality of OCaml packages in general, including those that are available through Guix.

Some very productive hack session resulted in a new Guix package importer for Opam packages and various fixes for the Opam package in Guix. The Opam website now also includes instructions on how to install Opam with Guix.

“User stories”

The “user stories” session focused on a couple of questions: How can we put reproducible builds into the hands of users? How do they benefit from it? This has been an on-going theme in Reproducible Builds Summits. We shared ideas about this before: users should be able to choose their provider of package binaries, it should be easy for them to (re)build packages locally, and it should be easy for them to challenge servers that publish binaries. Guix satisfies these requirements but could probably go further. These and other ideas were explored. In particular it was interesting to see several distros work on setting up rebuilders that would allow them to have more diversity and thus more confidence in the binaries that they ship.

Distributing packages over IPFS

Developers of IPFS offered a session on package distribution over IPFS. IPFS implements peer-to-peer unencrypted data storage; there are privacy and censorship-resistance issues that it leaves unaddressed, but its efficiency and convenience are appealing for the distribution of package binaries. An IPFS package landed in Guix right in time for the summit, which allowed us to play with it. Together with fellow Nix hackers, we sketched an incremental path towards IPFS integration in Guix and Nix. We hope to be able to deliver a first implementation of that in Guix soon—stay tuned!

Java and other JVM languages

The Java reproducibility working group discussed what was needed to reproduce builds of Java packages. We focused mainly on reproducing builds with Maven and realized that differences in vocabulary affect our goals. For instance, Maven developers don't consider plugins to be dependencies for the build, while in the context of Guix anything contributing to the build is declared as an input. Our interactions with other projects throughout the summit has led to similar revelations and contributed to improved mutual understanding.

In Guix we have packages for two more JVM languages other than Java itself: Groovy and Clojure. Other JVM languages, such as Scala and Kotlin (both dependencies of Gradle), however, come with additional challenges. The Scala compiler, for example, is written in Scala, and although there are other implementations of the compiler, they are also written in Scala. Julien started writing a bootstrap compiler for Scala in Java, which is already able to produce an AST for Scala files and produce JVM bytecode. In cases like this where it is not feasible to retrace historic steps to bootstrap a language, writing a new compiler implementation may surprisingly be the most reasonable thing to do.

Bootstrapping

We were excited to see increased interest in bootstrapping, which may have been sparked by recent major successes in the early Scheme/C bootstrap for GNU+Linux systems. While a few of us successfully hunted down a confusing miscompilation bug, parts of the early C bootstrap were ported to Nix, demonstrating that the bootstrapping work that started in the context of Guix can benefit other GNU+Linux distributions with minor adaptations.

With the addition of Gash as a Bash and GNU coreutils replacement on the horizon, we are looking forward to another dramatic reduction of the "binary seed" used to bootstrap the GNU system --- first via Guix and hopefully for other free software system distributions in the future.

Thanks!

We would like to thank the organizers who helped make the Reproducible Build Summit a productive, enjoyable and friendly workshop: Beatrice and Gunner of Aspiration, Holger, Nicolas, Vagrant, Chris and Mattia.

About GNU Guix

GNU Guix is a transactional package manager for the GNU system. The Guix System Distribution or GuixSD is an advanced distribution of the GNU system that relies on GNU Guix and respects the user's freedom.

In addition to standard package management features, Guix supports transactional upgrades and roll-backs, unprivileged package management, per-user profiles, and garbage collection. Guix uses low-level mechanisms from the Nix package manager, except that packages are defined as native Guile modules, using extensions to the Scheme language. GuixSD offers a declarative approach to operating system configuration management, and is highly customizable and hackable.

GuixSD can be used on an i686, x86_64, ARMv7, and AArch64 machines. It is also possible to use Guix on top of an already installed GNU/Linux system, including on mips64el and aarch64.