9.1 Setting Up a Head Node

The recommended approach is to set up one head node running guix-daemon and exporting /gnu/store over NFS to compute nodes.

Remember that guix-daemon is responsible for spawning build processes and downloads on behalf of clients (see Invoking guix-daemon in GNU Guix Reference Manual), and more generally accessing /gnu/store, which contains all the package binaries built by all the users (see The Store in GNU Guix Reference Manual). “Client” here refers to all the Guix commands that users see, such as guix install. On a cluster, these commands may be running on the compute nodes and we’ll want them to talk to the head node’s guix-daemon instance.

To begin with, the head node can be installed following the usual binary installation instructions (see Binary Installation in GNU Guix Reference Manual). Thanks to the installation script, this should be quick. Once installation is complete, we need to make some adjustments.

Since we want guix-daemon to be reachable not just from the head node but also from the compute nodes, we need to arrange so that it listens for connections over TCP/IP. To do that, we’ll edit the systemd startup file for guix-daemon, /etc/systemd/system/guix-daemon.service, and add a --listen argument to the ExecStart line so that it looks something like this:

ExecStart=/var/guix/profiles/per-user/root/current-guix/bin/guix-daemon --build-users-group=guixbuild --listen=/var/guix/daemon-socket/socket --listen=0.0.0.0

For these changes to take effect, the service needs to be restarted:

systemctl daemon-reload
systemctl restart guix-daemon

Note: The --listen=0.0.0.0 bit means that guix-daemon will process all incoming TCP connections on port 44146 (see Invoking guix-daemon in GNU Guix Reference Manual). This is usually fine in a cluster setup where the head node is reachable exclusively from the cluster’s local area network—you don’t want that to be exposed to the Internet!

The next step is to define our NFS exports in /etc/exports by adding something along these lines:

/gnu/store    *(ro)
/var/guix     *(rw, async)
/var/log/guix *(ro)

The /gnu/store directory can be exported read-only since only guix-daemon on the master node will ever modify it. /var/guix contains user profiles as managed by guix package; thus, to allow users to install packages with guix package, this must be read-write.

Users can create as many profiles as they like in addition to the default profile, ~/.guix-profile. For instance, guix package -p ~/dev/python-dev -i python installs Python in a profile reachable from the ~/dev/python-dev symlink. To make sure that this profile is protected from garbage collection—i.e., that Python will not be removed from /gnu/store while this profile exists—, home directories should be mounted on the head node as well so that guix-daemon knows about these non-standard profiles and avoids collecting software they refer to.

It may be a good idea to periodically remove unused bits from /gnu/store by running guix gc (see Invoking guix gc in GNU Guix Reference Manual). This can be done by adding a crontab entry on the head node:

root@master# crontab -e

... with something like this:

# Every day at 5AM, run the garbage collector to make sure
# at least 10 GB are free on /gnu/store.
0 5 * * 1  /usr/local/bin/guix gc -F10G

We’re done with the head node! Let’s look at compute nodes now.