Cuirass 1.0 released

We are pleased to announce the release of Cuirass version 1.0, after almost five years of development and around 700 commits from 14 contributors.

Cuirass is the GNU Guix continuous integration software. It's a general purpose build automation server written in GNU Guile that checks out sources from VCS repositories, execute build jobs and store build results in a database. Cuirass also provides a web interface to monitor the build results.

Cuirass is running on the GNU Guix build farm.

Since January, the project is funded through the NGI0 PET Fund, a fund established by NLnet with financial support from the European Commission's Next Generation, as explained here.

Thanks to this support, we were able to speed up the developments and finally propose a first release of this software. Many things have changed in Cuirass over the years and now is the perfect time to give it a try.

Here are the highlights of this new release.

Database

Let's start with the database that is the core of Cuirass. Until recently Cuirass was using an SQLite3 database. This technological choice proved to be quite challenging, and we had some troubles to make it scale as discussed here.

Cuirass now uses a PostgreSQL database, bringing the performance issues to an end while providing much more stability. Almost all the SQL queries are covered by test cases.

Specifications

In order to build some derivations, Cuirass first needs to be told what to build. Originally, an obscure association list describing the requested build jobs had to be passed to Cuirass.

Cuirass now operates on specification records that are described here. This input format is much more easy to understand for the user. It relies on Guix channels, which are well-adopted.

Here are a few different specifications examples.

This will build all the packages of the my-channel channel.

(list (specification
       (name "my-channel")
       (build '(channels my-channel))
       (channels
        (cons (channel
               (name 'my-channel)
               (url "https://my-channel.git"))
              %default-channels))))

This will build the linux-libre package on the default Guix channel master branch.

(list (specification
           (name "my-linux")
           (build '(packages "linux-libre"))))

Finally, this will build the packages declared in the my-manifest file of the my-channel channel, against the core-updates branch of the default Guix channel.

(list (specification
       (name "my-core-manifest")
       (build '(manifests "my-manifest"))
       (channels
        (list (channel
               (name 'my-channel)
               (url "https://my-channel.git"))
              (channel
               (name 'guix)
               (url %default-channel-url)
               (branch "core-updates"))))))

For people willing to spare some parens, a specification edition form has been implemented in the Web interface.

Specification creation

The Cuirass home page has also been updated to reflect this new input format.

Notifications

This feature was rightfully requested many times as this is a basic of any respectable CI system. Cuirass can now report failing and fixed builds in three different ways:

New build mode

The traditional way of building things in Cuirass is to send batches of derivations that need to be built to the local Guix daemon. The daemon can possibly offload those builds to other machines. While it's probably the most sensible way to proceed, this solution doesn't scale well and suffers from some limitations.

For all those reasons, using a new build mode seemed like a necessary evil. The rationale behind this new build mode is to have Cuirass communicate directly with the Guix daemons of all the offloading machines. Instead of dealing with a single, local, Guix daemon, Cuirass can now interact with several Guix daemons on remote machines.

The build jobs are not submitted to the local Guix daemon. Instead, a remote server dispatches build requests to the connect remote workers, according to the build priorities.

The remote server and the connected workers communicate using ZeroMQ over TCP. The workers are able to discover the remote server using Avahi.

The built items are exchanged as substitutes by spawning Guix publish servers both on the remote server and on each connected remote worker.

It seems more complex, and it is indeed more complex. However, the performance gains are real.

Build machines CPU idle percentage

This chart shows the CPU idle time percentage of the GNU Guix build farm machines. The introduction of the remote building mechanism around January 2021 results in a much higher activity of the connected machines.

This remote build mode also unlocked new features such as:

The workers status page is accessible here.

Workers status page

The machine status page is accessible here.

Machine status page

Web interface

Besides the features related to the specification record introduction, several improvements were brought to the Web interface.

Some administration actions that previously required manual SQL intervention can now be performed directly through the Web interface.

Any Cuirass administrator can now:

The build page was also improved to display the build weather and a build history.

Build page

Several issues were also fixed such as the broken pagination and the negative build duration.

Metrics

Cuirass computes periodically various metrics such as:

Metrics

Those metrics can be browsed here.

Documentation

The Cuirass documentation is now updated to reflect those changes and can be browsed online.

The release itself is available on Cuirass home page.

The Guix's Cuirass package as well as the Cuirass service were also updated.

Going further

The NLNet grant will allow me to keep working on Cuirass for a couple more months. This will hopefully help us to:

This release is an important milestone as, combined with the recent substitute improvements, the whole substitute availability & download speed situation is now largely mitigated, at least on Intel architectures.

Don't hesitate to run your own Cuirass server to build stuff ahead of the GNU Guix build farm, or to build your custom channels. Also feel free to share the features you would like to see in the next Cuirass release.

Sujets liés :

CI Cuirass Releases