Guix

Guix User and Contributor Survey 2024: The Results (part 3)

Steve George — 28. januari, 2025

Today we're looking at the results from the Contributor section of the Guix User and Contributor Survey (2024). The goal was to understand how people contribute to Guix and their overall development experience. A great development experience is important because a Free Software project's sustainability depends on happy contributors to continue the work!

See Part 1 for insights about Guix adoption, and Part 2 for users overall experience. With over 900 participants there's lots of interesting insights!

Contributor community

The survey defined someone as a Contributor if they sent patches of any form. That includes changes to code, but also other improvements such as documentation and translations. Some Guix contributors have commit access to the Guix repository, but it's a much more extensive group than those with commit rights.

Of the survey's 943 full responses, 297 participants classified themselves as current contributors and 58 as previous contributors, so 355 participants were shown this section.

The first question was (Q22), How many patches do you estimate you've contributed to Guix in the last year?

Note that the percentages in this table, and throughout the posts, are rounded up to make them easier to refer to.

The percentage is the percentage of contributors that sent patches in the last year. That means the 42 participants who were previous contributors have been excluded.

Figure 13 shows this visually:

As we can see many contributors send a few patches (61%), perhaps updating a package that they personally care about. At the other end of the scale, there are a few contributors who send a phenomenal number of patches.

Active contributors

It's interesting to investigate the size of Guix's contributor community. While running the survey I did some separate research to find out the total number of contributors. I defined an Active contributor as someone who had sent a patch in the last two years, which was a total of 454 people. I deduplicated by names, but as this is a count by email address there may be some double counting.

This research also showed the actual number of patches that were sent by contributors:

Figure 14 shows this:

Together this give us an interesting picture of the contributor community:

• There's a good community of active contributors to Guix: 300 in the survey data, and 454 from the direct research.
• A significant percentage of contributors send one, or a few patches. This reflects that packaging in Guix can be easy to get started with.
• The direct research shows an even distribution of contributors across the different levels of contribution. This demonstrates that there are some contributors who have been working on Guix for a long-time, as well as newer people joining the team. That's great news for the sustainability of the project!
• There are also some very committed contributors who have created a lot of patches and been contributing to the project for many years. In fact, the top 10 contributors have all contributed over 700 patches each!

Types of contribution

The survey also asked contributors (Q23), How do you participate in the development of Guix?

Figure 15 shows this as a pie chart (upping my game!):

Of course, the same person can contribute in multiple areas: as there were 531 responses to this question, from 355 participants, we can see that's happening.

Complex projects like Guix need a variety of contributions, not just code. Guix's web site needs visual designers who have great taste, and certainly a better sense of colour than mine! We need documentation writers to provide the variety of articles and how-tos that we've seen users asking for in the comments. The list goes on!

Unsurprisingly, Guix is code heavy with 60% of contributors focusing in this area, but it's great to see that there are people contributing across the project. Perhaps there's a role you can play? ... yes, you reading this post!

Paid vs unpaid contribution

FOSS projects exist on a continuum of paid and unpaid contribution. Many projects are wholly built by volunteers. Equally, there are many large and complex projects where the reality is that they're built by paid developers — after all, everyone needs to eat!

To explore this area the survey then asked (Q24), Are you paid to contribute to Guix?

The results show:

We can see this as Figure 16 :

Some thoughts:

• Guix is a volunteer driven project.
• The best way to work on Guix professionally is to find a way to make it part of your employment.
• For everyone involved in the project the fact that the majority of contributors are doing it in their spare time has to be factored into everything we do, and how we treat each other.

Previous contributors

Ensuring contributors continue to be excited and active in the project is important for it's health. Ultimately, fewer developers means less can be done. In volunteer projects there's always natural churn as contributor's lives change. But, fixing any issues that discourages contributors is important for maintaining a healthy project.

Question 25 was targeted at the 59 participants who identified themselves as Previous Contributors. It asked, You previously contributed to Guix, but stopped, why did you stop?

The detailed results are:

Figure 17 shows this graphically:

There were 80 answers from the 59 participants so some participants chose more than one reason.

• As we can see a change in external circumstances was the biggest reason and to be expected.
• The next reason was Response to contributions was slow and/or reviews arduous, as we'll see this repeatedly showed-up as the biggest issue.
• Next was The contribution process (e.g. email and patch flow) which also appears in many comments. Judging by the comments the email and patch flow may be a gateway factor that puts-off potential contributors from starting. There's no way for the survey to determine this as it only covers people that started contributing and then stopped, but the comments are interesting.

Future contributions

Q26 asked contributors to grade their likelihood of contributing further, this is essentially a satisfaction score.

The question was, If you currently contribute patches to Guix, how likely are you to do so in the future?

Figure 18 shows this graphically:

Out of the audience of current and previous contributors, 355 in total:

• The 35% of contributors who are 'Certain' they'll contribute is a great sign.
• The 31% that are 'Likely' shows that there's a good pool of people who could be encouraged to continue to contribute.
• We had 58 participants who categoried themselves as Previous Contributors and 41 answered this question with definitely or probably not, that's about 12%. That leaves the 80 (23%) who are loosely positive.

Improving contribution

The survey then explored areas of friction for contributors. Anything that reduces friction should increase overall satisfaction for existing contributors.

The question (Q27) was, What would help you contribute more to the project?

Figure 19 bar chart visualises this:

The 355 contributors selected 933 options for this question, so many of them selected multiple aspects that would help them to contribute more to the project.

Conclusions we can draw are:

• Ensuring there's Timely reviews and actions taken on contributions is the biggest concern for contributors, and as we saw also causes contributors to become demoralised and cease working on the project.
• The concern over both Debugging and error messages has been a consistent concern from contributors.
• Interestingly, documentation of Guix's internals is a priority in this list, but in other questions it doesn't appear as a high priority.

Comments on improving contribution

Jumping ahead, the last question of the contributor section (Q30) was a comment box. It asked, Is there anything else that you would do to improve contributing to Guix?

The full list of comments from Q27, and Q30 are available and worth reading (or at least scanning!).

Looking across all of them I've created some common themes - picking a couple of example comments to avoid repetition:

• Compensation for developers: there were comments from developers who want to work on Guix professionally, or people offering to donate.
• "[Part of a long comment] ... For me personally it really boils down to the review process. Some patches just hang there for many months without *any* reaction. That is quite discouraging to be honest. So if there would be fund raising, I think it should (for a large part) go to paying someone (maybe even multiple people?) to do code reviews and merge patches. And in general do the "gardening job" on the issue tracker."
• "I would be happy to contribute to some kind of fund, maybe by a monthly subscription, which would award stipends for experienced guix contributors to work on patch review."
• Complexity of contribution: where the overall set of steps required to contribute were too complex.
• "For occasional contributions, the threshold is higher than for most projects, in part due to less common tools used in the project (bugtracker for example)"
• "[long comment where the substance is] I'd rather spend my limited time contributing to a 100% free software project than reading 20 webpages on how to use all the CLI tooling."
• Issues with email-based contribution: concerns about the steps to create a patch, email it and so forth.
• "Difficult; I am not used to the email workflow and since I'm not contributing often it is close to rediscovering everything again which is annoying. There isn't a specific thing that could solve that I guess. apologies if this doesn't say much"
• "The GNU development process with mailing lists and email patches is the most difficult aspect."
• Issues with speed and capacity of patch reviews: this is the highest priority amongst contributors, so there were many comments about patches not being reviewed, or reviews taking a long time.
• "I really dislike that 70% of my patches don't get reviewed at all, how simple or trivial they may be. I do really test them and dogfood so contributing seems like a waste of time as someone without commit-access."
• "I already checked "timely reviews/actions", but I want to emphasize how demoralizing my contribution experience was. I was excited about how easy it was to make, test, and submit a patch; I would love to help improve the packaging situation for things that I use. But it's been about a year now since I submitted some patches and have received exactly 0 communication regarding what I submitted. No reviews, no comments, no merge, nothing. Really took the wind out of my sails"
• Automate patch testing and acceptance: suggestions to speed up the review pipeline by automating.
• "A bias for action. If no one shows interest in a patch, and it's constrained, it should just be landed."
• "Minimizing the required work needed to keep packages up to date. Most of the best developers in Guix are commiters and all the time they have to spend reviewing package update patches etc. is away from improving Guix's core. They should be able to focus on things like shepherd, bootloader configuration, guix-daemon in guile, distributed substitutes or a more modular system configuration (e.g. letting nginx know of certbot certificates without having to manually pass (ssl-certificate "/etc/...")).*"
• Adding more committers: comments that more contributors would increase project velocity, and general concerns about how difficult it is to become a committer.
• "Keep manual up to date, I think we need more committers doing reviews and give more love to darker corners."
• "All the best. The project might need more hands to review incoming patches."
• Addition of a pull requests workflow: specific comments requesting the addition of a Forge experience.
• "I would use Forgejo (either an instance or at codeberg) to simplify contributions and issue handling. In my humble and personal opinion the forge workflow makes it much easier to get an overview of what is going on and to interact with others on issues and PRs"
• "I think opening a pull request approach would really modernize the way of working and help welcome more people. We could still easily support patches too."
• Automating package builds and tests: comments relating to automation of building packages as part of the contribution flow.
• "We really need to improve the CICD situation. I see we have so many system tests that could catch issues. Let's make sure each patch has run at least against a couple of those system tests before it is being merged, or even before a reviewer has even looked at. Today a colleague of mine, who is just getting into Guix because I told him had issues with the u-boot-tools package not being built on a substitute server and being broken. Yeah, that can happen, but it happens all the time and it is such a bad experience for new and existing users."
• Bugtracker improvements: comments about improving the bug tracker.
• "A formal method to count the number of users affected by an issue so that developers know what to prioritize. For example, ubuntu's launchpad has a "bug heat" metric which counts the number of users that report they are affected by the bug."
• Debugging and error reporting: challenges debugging issues due to difficult error messages in Guix, or underlying Guile weaknesses.
• "The development workflow with Guile. I've recently switched to arei/ares, but I'm still a total newbie on how to effectively develop and navigate. I've used quite some Common Lisp, and I have my own channel with a handful packages, but it takes a long time to develop without the necessary knowledge of both Guile setup and Guix setup."
• "I just want to reiterate that the debugging process can be painful sometimes. Sometimes guile gives error messages that can be bewildering. As an example, I spent awhile debugging the error message "no value specified for service of type 'myservice'". The problem was that I omitted the default-value field in my service-type, but the error message could have included the default-value field."
• Runtime performance and resource usage: where it makes the experience of building and testing Guix slow or unusable.
• "Foremost faster guix evals. I forget what I was doing while it runs."
• "Building guix takes too long time for packagers. It is not clear why everything needs to be compiled when only contributing a package. Why does the documentation need to be built when adding a package?"
• Practical guides, how-tos and examples: requests for direct instructions or examples, as compared to reference documentation.
• "Improve the documentation on how to contribute. It is currently very hard to follow, some sections are simply in the wrong order, others presuppose the reader wants to evaluate several different alternatives instead of pointing to one simple way of doing things. And steps that though simple are unusual and will seem complicated to most people don't get explained in sufficient detail and examples."
• FSF association as a constraint: concerns about Free Software and GNU as an organisation constraining practical user freedom.
• "*Drop GNU and drop the hardline stance against discussing any proprietary software. It doesn't have to be supported directly, but at least have a link to Nonguix or something. Or have a feature flag like Nixpkgs. Who cares if the distro is certified by an organization that is pretty much irrelevant, actually giving people agency over their tech is what should be the number one goal."
• "Guix is one of the GNU projects with the most potential and relevance, but unfortunately it seems association with the FSF is a contributing factor to limited adoption."
• Not enough FSF: comments that the Guix project was not sufficiently supportive of FSF and/or Richard Stallman.
• "collaborate more with other GNU projects"
• Commit messages: concerns that the commit message format is repetitious or unneccessary.
• "Encourage or enforce the usage of commit messages that detail why a change is done (and not what is done - which is already visible from the git diff)."
• Importers and language ecosystem: comments about possible improvements to deal with dynamic language ecosystems (e.g. Javascript and Rust).
• "Improved build systems and importers. Generally improving management of high-noise ecosystems (Maven, Rust, NPM, …)"
• "Packaging Golang or Rust apps can be difficult and time-consuming because Guix requires all (recursive) dependencies to be packaged in Guix. I often gave up and just re-packaged a precompiled binary from upstream or another distro. It would be much easier if Guix relied on existing language-specific dependency management (e.g., use Cargo.lock files to fix all dependencies) - not perfect from Guix pov, but pragmatic and much more usable."
• "More flexible package definitions but also more strict filtering of available packages. For example, allow some packages to use the internet in the build path (so you may easily install pip packages like TensorFlow, Flax), but by default do not allow installation of NonFree licenses and network enabled packages. We allow package transformations (--with-commit) which need network access anyway and doesn't verify hashes, I think this can be allowed. The end goal of a package should be to be reproducible from source, but the current goal can be usability, widespread adoption, reliability. This way we can start to use Guix in more scientific experiments and super computers, then the new users can help contribute further."
• Project communications methods: requests for communications within the project to use modern methods (e.g. Matrix, Discourse, Github).
• "Having a Discourse instance, so that people can ask questions and others and chime in and the best answers get upvotes. IRC and mailing lists are suboptimal. My success rate of getting ANY reply to my question have been consistently less than 50% regardless of the time of the day, because in IRC it scrolls down and questions go out of focus. Also in IRC the threads of discussion is getting mixed. Keep the IRC, but provide a Discourse instance. I personally even pay for paart of the cost."
• Repo organisation: ideas to widen the set of contributors by having a community repo (e.g. Arch Linux like).
• "I would like more packages under Guix, but I am not convinced that adding them all to the Guix channel is the way. I believe a large number of Guix packages should be moved to guix-free or similar channel. The packages in Guix itself should be the minimal ones that come installed in Guix system. The guix-free channel should be part of default-channels."
• "I feel like channels are a cumbersome alternative to community packages. I previously tried to package a lesser known programming language compiler for Guix but never got replies to my patches to contribute the package. Perhaps there could be a core and community channel with stronger/weaker standards."
• Project culture: concerns about the project being inward looking, not inclusive and with too much gatekeeping. Most comments in this area were very passionate, and in some cases a bit angry.
• "TODO lists and direction is very helpful. Lists of "good first task" or "very important — need help with" etc, things to motivate others to contribute in. Also helpful if people ACTUALLY become part of the distro and it's not all gate-kept by idiots with attitude. I don't want to invest 1000 man hours to prove myself worthy of maintanership of a package!"

Organisational and social improvements

It's common in FOSS projects to focus on the technical issues, but Free Software is a social endeavour where organisational and social aspects are just as important. Q28 focused on the social and organisational parts of contribution by asking, What organisational and social areas would you prioritise to improve Guix?

This was a ranked question where participants had to prioritise their top 3. The rationale for asking it in this way was to achieve prioritisation.

It's useful to look at the results in two ways, first the table where participants set their highest priority (Rank 1):

Out of the 355 participants in this section, 338 answered this question and marked their highest priority.

Figure 20 shows it as a pie chart:

This second table shows how each category was prioritied across all positions:

Figure 21 shows this as a stacked bar chart. Each of the categories is the position for a rank (priority), so the smallest overall priority is the most important:

Looking at these together:

• It's clear that the highest priority (table 28) is to Improve the speed and capacity of the contribution process, as 63% of participants selected it and nothing else was close to it.
• I found it quite confusing that it didn't also score highly in the second and third rank questions, which negatively impacts the overall score. This seems to be caused by the question having a significant drop-off in answers: 338 participants set their 'Rank 1', but only 264 set a 'Rank 2' and then 180 set a 'Rank 3'. The conclusion I draw is that for many contributors the sole important organisational improvement is to improve the speed and capacity of the contribution process.
• Nonetheless, overall Project decision making and co-ordination was the most important social improvement across all ranks, and it was the second most important one for 'Rank 1' — so that's pretty consistent. Other than improving the contribution process this was the next most important item on contributors minds.
• Promotion and advocacy also seems to be important, though there are very few comments about it in the survey overall. The next most important across all ranks was Fund raising, which does get some comments.

Technical improvements

The partner question was Q29 which asked, What technical areas would you prioritise to improve Guix overall?

This was also a ranked question where participants had to prioritise their top 3.

There were 345 answers for the highest priority, 327 for the second rank and 285 for the third rank — so not as significant a drop-off as for the social question. Figure 22 shows this as a bar chart:

As before I've converted them to priorities in each rank. The smallest overall score is the highest priority:

Figure 23 shows this as a stacked bar chart.

Some things that are interesting from this question:

• For the technical improvements there isn't a single over-riding 'Rank 1' priority (table 30). The first choice, Debugging and error reporting, does come up consistently in comments as a problem for packagers, and across all three ranks it's the third priority.
• Across all ranks Debugging and error reporting along with Runtime performance (speed and memory) are high priorities. These are probably quite connected as there's lots of comments in the survey about error reporting and slow evaluations making development time-consuming and difficult.
• It's possible to think of the second and third priorities for 'Rank 1' (table 30) as being connected, since the velocity needed for Making the latest version of packages available would be helped by Automate patch testing and acceptance. We can see from the second table that through all priorities this is the area that contributors care about the most.
• We asked the same question of all Users (Q21) earlier in the survey. This time the question was for Contributors only and there were a few specific contribution-focused options. It's interesting to see the contrast between contributors and users priorities:
• For both the contributor (P2) and users (P1) improving the runtime performance was a high priority, so it's pretty consistent.
• For users making Guix easier to learn was the second highest priority, there wasn't really an equivalent option in the contributor question.
• Users identified Making the latest versions of packages available (package freshness) as very important and it's also a high priority in the first rank for contributors. However, overall it was middle of the pack for them — with both Project infrastructure (e.g. continuous integration) and Contribution workflow (e.g. Pull Requests) coming higher.

Key insights recap

That completes our review of the contributor section! Here are the key insights I draw:

1. The size of the active contributor community (~450) is really exciting. Many developers send a few patches (~60%), while at the other end of the scale there are some who have sent hundreds.
2. Retaining and developing contributors is important for the project's sustainability. About 66% of active developers are likely to contribute again. That's great, how can we encourage that to happen?
3. The key reasons contributors stopped (aside from life changes) was a slow response to contributions and the contribution process (e.g email and patch flow).
4. Improving the capacity and speed of reviews was also the over-riding concern for active contributors by a significant margin. High priority suggestions were automating patch testing and acceptance, along with improving the projects infrastructure (e.g. continuous integration).
5. Technical improvements to the developer experience were improving debugging and error reporting, runtime performance and also providing a more commonly used contribution process (e.g. Pull Requests).
6. Finally, the project is 95% a volunteer one, so we should bear in mind that everyone's contributing to Guix on their personal time! While it's great to see all this fantastic feedback and it's very useful, Guix is a collective of volunteers with the constraints that brings.

Getting the Data

We've really squeezed the juice from the lemon over these three posts — but maybe you'd like to dig into the data and do your own analysis? If so head over to the Guix Survey repository where you'll find all the data available to create your own plots!