A build daemon in Guile

Christopher Baines — October 30, 2023

When using Guix, you might be aware of the daemon. It runs in the background but it's a key component in Guix. Whenever you've been using Guix to operate on the store, whether that's building something or downloading some substitutes, it's the daemon managing that operation.

The daemon also is a key part of the history of Guix. The Guix project started mixing Guile with ideas from the Nix project, and the guix-daemon is a fork of the nix-daemon with some tweaks made over the years. Rather than being implemented in Guile though, the daemon is implemented in C++ with some helpers written in Guile. Given the focus on Guile in Guix, this is unusual, and I believe it's made working on the daemon less desirable, especially since rewriting it in Guile has been discussed for many years now. It has been the topic of a Google Summer of Code internship by Caleb Ristvedt back in 2017, which helped clarify implementation details and led to some preliminary code.

What would a build daemon in Guile bring?

Guix already has code written in Guile for doing some of what the daemon does internally, so being able to use this Guile code inside and outside the daemon would simplify Guix and allow removing the C++ code.

There isn't Guile code yet for everything the daemon does though, so getting to this point will make new exciting features easier to implement. That could be things like making it easier to use Guix in environments where running the daemon in the usual way is inconvenient or infeasible. It may also help with portability, so help with running Guix on the Hurd and new architectures.

As someone who's more experienced writing Guile than C++, I'm also hoping it'll generally make hacking on the daemon more accessible. This in turn might lead to new features. For example, I think having a build daemon written in Guile will simplify implementing a way to jump in to a build and inspect the environment.

With that in mind, I'm excited to announce that support from NLNet, will allow me to focus for the next year on getting a Guile implementation of the build daemon written and adopted.

A technical plan

Building on the recent discussion of this topic on the guix-devel@gnu.org mailing list, here's some technical thoughts on how I'm approaching this.

While I think there's a substantial amount of work to do, progress towards a Guile guix-daemon has already been made. Given that things in Guix have probably changed since this work has happened, I plan to carefully review that existing work (most of which can be found on the guile-daemon branch.

The priority for the Guile daemon is backwards compatibility, so the plan is to allow switching between the C++ implementation and Guile implementation without any issues. This'll require not making changes to the database schema, and generally doing things in a way which the current C++ daemon will understand.

Like the Guix Build Coordinator, I'm planning to make the daemon a single process using Fibers for concurrency. This is in contrast to the forking model using by the C++ daemon. Even though it's not a priority to address feature issues with the current daemon, this approach might help to reduce database contention issues experienced with the current daemon, and allow for less locking, like not having the big GC lock for example.

I'm planning on publishing more blog posts as the project progress, so keep an eye on the Guix blog for future updates.

Acknowledgments

Thanks to Simon Tournier and Ludovic Courtès for providing feedback on an earlier draft of this post.

About GNU Guix

GNU Guix is a transactional package manager and an advanced distribution of the GNU system that respects user freedom. Guix can be used on top of any system running the Hurd or the Linux kernel, or it can be used as a standalone operating system distribution for i686, x86_64, ARMv7, AArch64 and POWER9 machines.

In addition to standard package management features, Guix supports transactional upgrades and roll-backs, unprivileged package management, per-user profiles, and garbage collection. When used as a standalone GNU/Linux distribution, Guix offers a declarative, stateless approach to operating system configuration management. Guix is highly customizable and hackable through Guile programming interfaces and extensions to the Scheme language.