https://guix.gnu.org/feeds/blog/dissecting-guix.atomGNU Guix — Blog — Dissecting Guixfeed author nameGNU Guixhttps://guix.gnu.org/themes/initial/img/icon.png2024-03-20T10:57:57Zhttps://guix.gnu.org/blog/2023/dissecting-guix-part-3-g-expressions//Dissecting Guix, Part 3: G-Expressions(2023-04-19T12:30:00Z2023-04-19T12:30:00Z Welcome back to Dissecting Guix !
Last time, we discussed monads ,
the functional programming idiom used by Guix to thread a store connection
through a series of store-related operations. Today, we'll be talking about a concept rather more specific to Guix:
g-expressions . Being an implementation of the Scheme language, Guile is built
around s-expressions , which can
represent, as the saying goes, code as data , thanks to the simple structure of
Scheme forms. As Guix's package recipes are written in Scheme, it naturally needs some way to
represent code that is to be…<p>Welcome back to <a href="https://guix.gnu.org/en/blog/tags/dissecting-guix">Dissecting Guix</a>!
Last time, we discussed <a href="https://guix.gnu.org/en/blog/2023/dissecting-guix-part-2-the-store-monad">monads</a>,
the functional programming idiom used by Guix to thread a store connection
through a series of store-related operations.</p><p>Today, we'll be talking about a concept rather more specific to Guix:
<em>g-expressions</em>. Being an implementation of the Scheme language, Guile is built
around <a href="https://en.wikipedia.org/wiki/S-expression"><em>s-expressions</em></a>, which can
represent, as the saying goes, <em>code as data</em>, thanks to the simple structure of
Scheme forms.</p><p>As Guix's package recipes are written in Scheme, it naturally needs some way to
represent code that is to be run only when the package is built. Additionally,
there needs to be some way to reference dependencies and retrieve output paths;
otherwise, you wouldn't be able to, for instance, create a phase to install a
file in the output directory.</p><p>So, how do we implement this "deferred" code? Well, initially Guix used plain
old s-expressions for this purpose.</p><h1>Once Upon a Time</h1><p>Let's say we want to create a store item that's just a symlink to the
<code>bin/irssi</code> file of the <code>irssi</code> package. How would we do that with an
s-expression? Well, the s-expression itself, which we call the <em>builder</em>, is
fairly simple:</p><pre><code class="language-scheme">(define sexp-builder
`(let* ((out (assoc-ref %outputs "out"))
(irssi (assoc-ref %build-inputs "irssi"))
(bin/irssi (string-append irssi "/bin/irssi")))
(symlink bin/irssi out)))</code></pre><p>If you aren't familliar with the "quoting" syntax used to create s-expressions,
I strongly recommend that you read the excellent Scheme Primer; specifically,
section 7, <a href="https://spritely.institute/static/papers/scheme-primer.html#scheme-lists-and-cons"><em>Lists and
"cons"</em></a>
and section 11, <a href="https://spritely.institute/static/papers/scheme-primer.html#scheme-extensibility"><em>On the extensibility of Scheme (and Lisps in
general)</em></a></p><p>The <code>%outputs</code> and <code>%build-inputs</code> variables are bound within builder scripts to
<em>association lists</em>, which are lists of pairs that act like key/value stores,
for instance:</p><pre><code class="language-scheme">'(("foo" . "bar")
("floob" . "blarb")
("fvoolag" . "bvarlag"))</code></pre><p>To retrieve values from association lists, which are often referred to as
<em>alists</em>, we use the <code>assoc-ref</code> procedure:</p><pre><code class="language-scheme">(assoc-ref '(("boing" . "bouncy")
("floing" . "flouncy"))
"boing")
⇒ "bouncy"</code></pre><p><code>%outputs</code>, as the name might suggest, maps derivation output names to the paths
of their respective store items, the default output being <code>out</code>, and
<code>%build-inputs</code> maps inputs labels to their store items.</p><p>The builder is the easy part; we now need to turn it into a derivation and tell
it what <code>"irssi"</code> actually refers to. For this, we use the
<code>build-expression->derivation</code> procedure from <code>(guix derivations)</code>:</p><pre><code class="language-scheme">(use-modules (guix derivations)
(guix packages)
(guix store)
(gnu packages guile)
(gnu packages irc))
(with-store store
(let ((guile-3.0-drv (package-derivation store guile-3.0))
(irssi-drv (package-derivation store irssi)))
(build-expression->derivation store "irssi-symlink" sexp-builder
#:guile-for-build guile-3.0-drv
#:inputs `(("irssi" ,irssi-drv)))))
⇒ #<derivation /gnu/store/…-irssi-symlink.drv => /gnu/store/…-irssi-symlink …></code></pre><p>There are several things to note here:</p><ul><li>The inputs <em>must</em> all be derivations, so we need to first convert the packages
using <code>package-derivation</code>.</li><li>We need to explicitly set <code>#:guile-for-build</code>; there's no default value.</li><li>The <code>build-expression->derivation</code> and <code>package-derivation</code> procedures are
<em>not</em> monadic, so we need to explicitly pass them the store connection.</li></ul><p>The shortcomings of using s-expressions in this way are numerous: we have to
convert everything to a derivation before using it, and <em>inputs are not an
inherent aspect of the builder</em>. G-expressions were designed to overcome these
issues.</p><h1>Premortem Examination</h1><p>A g-expression is fundamentally a record of type <code><gexp></code>, which is, naturally,
defined in <code>(guix gexp)</code>. The two most important fields of this record type,
out of a total of five, are <code>proc</code> and <code>references</code>; the former is a procedure
that returns the equivalent s-expression, the latter a list containing
everything from the "outside world" that's used by the g-expression.</p><p>When we want to turn the g-expression into something that we can actually run as
code, we combine these two fields by first building any g-expression inputs that
can become derivations (leaving alone those that cannot), and then passing the
built <code>references</code> as the arguments of <code>proc</code>.</p><p>Here's an example g-expression that is essentially equivalent to our
<code>sexp-builder</code>:</p><pre><code class="language-scheme">(use-modules (guix gexp))
(define gexp-builder
#~(symlink #$(file-append irssi "/bin/irssi")
#$output))</code></pre><p><code>gexp-builder</code> is far more concise than <code>sexp-builder</code>; let's examine the syntax
and the <code><gexp></code> object we've created. To make a g-expression, we use the <code>#~</code>
syntax, equivalent to the <code>gexp</code> macro, rather than the <code>quasiquote</code> backtick
used to create s-expressions.</p><p>When we want to embed values from outside as references, we use <code>#$</code>, or
<code>ungexp</code>, which is, in appearance if not function, equivalent to <code>unquote</code>
(<code>,</code>). <code>ungexp</code> can accept any of four reference types:</p><ul><li>S-expressions (strings, lists, etc), which will be embedded literally.</li><li>Other g-expressions, embedded literally.</li><li>Expressions returning any sort of object that can be lowered into a
derivation, such as <code><package></code>, embedding that object's <code>out</code> store item; if
the expression is specifically a symbol bound to a buildable object, you can
optionally follow it with a colon and an alternative output name, so
<code>package:lib</code> is permitted, but <code>(get-package):lib</code> isn't.</li><li>The symbol <code>output</code>, embedding an output path. Like symbols bound to
buildable objects, this can be followed by a colon and the output name that
should be used rather than the default <code>out</code>.</li></ul><p>All these reference types will be represented by <code><gexp-input></code> records in the
<code>references</code> field, except for the last kind, which will become <code><gexp-output></code>
records. To give an example of each type of reference (with the return value
output formatted for easier reading):</p><pre><code class="language-scheme">(use-modules (gnu packages glib))
#~(list #$"foobar" ;s-expression
#$#~(string-append "foo" "bar") ;g-expression
#$(file-append irssi "/bin/irssi") ;buildable object (expression)
#$glib:bin ;buildable object (symbol)
#$output:out) ;output
⇒ #<gexp (list #<gexp-input "foobar":out>
#<gexp-input #<gexp (string-append "foo" "bar") …>:out>
#<gexp-input #<file-append #<package irssi@1.4.3 …> "/bin/irssi">:out>
#<gexp-input #<package glib@2.70.2 …>:bin>
#<gexp-output out>) …></code></pre><p>Note the use of <code>file-append</code> in both the previous example and <code>gexp-builder</code>;
this procedure produces a <code><file-append></code> object that builds its first argument
and is embedded as the concatenation of the first argument's output path and the
second argument, which should be a string. For instance,
<code>(file-append irssi "/bin/irssi")</code> builds <code>irssi</code> and expands to
<code>/gnu/store/…-irssi/bin/irssi</code>, rather than the <code>/gnu/store/…-irssi</code> that the
package alone would be embedded as.</p><p>So, now that we have a g-expression, how do we turn it into a derivation? This
process is known as <em>lowering</em>; it entails the use of the aptly-named
<code>lower-gexp</code> monadic procedure to combine <code>proc</code> and <code>references</code> and produce a
<code><lowered-gexp></code> record, which acts as a sort of intermediate representation
between g-expressions and derivations. We can piece apart this lowered form to
get a sense of what the final derivation's builder script would look like:</p><pre><code class="language-scheme">(define lowered-gexp-builder
(with-store store
(run-with-store store
(lower-gexp gexp-builder))))
(lowered-gexp-sexp lowered-gexp-builder)
⇒ (symlink
"/gnu/store/…-irssi-1.4.3/bin/irssi"
((@ (guile) getenv) "out"))</code></pre><p>And there you have it: a s-expression compiled from a g-expression, ready to be
written into a builder script file in the store. So, how exactly do you turn
this into said derivation?</p><p>Well, it turns out that there isn't an interface for turning lowered
g-expressions into derivations, only one for turning regular g-expressions into
derivations that first uses <code>lower-gexp</code>, then implements the aforementioned
conversion internally, rather than outsourcing it to some other procedure, so
that's what we'll use.</p><p>Unsurprisingly, that procedure is called <code>gexp->derivation</code>, and unlike its
s-expression equivalent, it's monadic. (<code>build-expression->derivation</code> and
other deprecated procedures were in Guix since before the monads system
existed.)</p><pre><code class="language-scheme">(with-store store
(run-with-store store
(gexp->derivation "irssi-symlink" gexp-builder)))
⇒ #<derivation /gnu/store/…-irssi-symlink.drv => /gnu/store/…-irssi-symlink …></code></pre><p>Finally, we have a g-expression-based equivalent to the derivation we earlier
created with <code>build-expression->derivation</code>! Here's the code we used for the
s-expression version in full:</p><pre><code class="language-scheme">(define sexp-builder
`(let* ((out (assoc-ref %outputs "out"))
(irssi (assoc-ref %build-inputs "irssi"))
(bin/irssi (string-append irssi "/bin/irssi")))
(symlink bin/irssi out)))
(with-store store
(let ((guile-3.0-drv (package-derivation store guile-3.0))
(irssi-drv (package-derivation store irssi)))
(build-expression->derivation store "irssi-symlink" sexp-builder
#:guile-for-build guile-3.0-drv
#:inputs `(("irssi" ,irssi-drv)))))</code></pre><p>And here's the g-expression equivalent:</p><pre><code class="language-scheme">(define gexp-builder
#~(symlink #$(file-append irssi "/bin/irssi")
#$output))
(with-store store
(run-with-store store
(gexp->derivation "irssi-symlink" gexp-builder)))</code></pre><p>That's a lot of complexity abstracted away! For more complex packages and
services, especially, g-expressions are a lifesaver; you can refer to the output
paths of inputs just as easily as you would a string constant. You do, however,
have to watch out for situations where <code>ungexp-native</code>, written as <code>#+</code>, would
be preferable over regular <code>ungexp</code>, and that's something we'll discuss later.</p><p>A brief digression before we continue: if you'd like to look inside a <code><gexp></code>
record, but you'd rather not build anything, you can use the
<code>gexp->approximate-sexp</code> procedure, which replaces all references with dummy
values:</p><pre><code class="language-scheme">(gexp->approximate-sexp gexp-builder)
⇒ (symlink (*approximate*) (*approximate*))</code></pre><h1>The Lowerable-Object Hardware Shop</h1><p>We've seen two examples already of records we can turn into derivations, which
are generally referred to as <em>lowerable objects</em> or <em>file-like objects</em>:</p><ul><li><code><package></code>, a Guix package.</li><li><code><file-append></code>, which wraps another lowerable object and appends a string to
the embedded output path when <code>ungexp</code>ed.</li></ul><p>There are many more available to us. Recall from the previous post,
<a href="https://guix.gnu.org/en/blog/2023/dissecting-guix-part-2-the-store-monad"><em>The Store Monad</em></a>,
that Guix provides the two monadic procedures <code>text-file</code> and <code>interned-file</code>,
which can be used, respectively, to put arbitrary text or files from the
filesystem in the store, returning the path to the created item.</p><p>This doesn't work so well with g-expressions, though; you'd have to wrap each
<code>ungexp</code>ed use of either of them with
<code>(with-store store (run-with-store store …))</code>, which would be quite tedious.
Thankfully, <code>(guix gexp)</code> provides the <code>plain-file</code> and <code>local-file</code> procedures,
which return equivalent lowerable objects. This code example builds a directory
containing symlinks to files greeting the world:</p><pre><code class="language-scheme">(use-modules (guix monads)
(ice-9 ftw)
(ice-9 textual-ports))
(define (build-derivation monadic-drv)
(with-store store
(run-with-store store
(mlet* %store-monad ((drv monadic-drv))
(mbegin %store-monad
;; BUILT-DERIVATIONS is the monadic version of BUILD-DERIVATIONS.
(built-derivations (list drv))
(return (derivation-output-path
(assoc-ref (derivation-outputs drv) "out"))))))))
(define world-greeting-output
(build-derivation
(gexp->derivation "world-greeting"
#~(begin
(mkdir #$output)
(symlink #$(plain-file "hi-world"
"Hi, world!")
(string-append #$output "/hi"))
(symlink #$(plain-file "hello-world"
"Hello, world!")
(string-append #$output "/hello"))
(symlink #$(plain-file "greetings-world"
"Greetings, world!")
(string-append #$output "/greetings"))))))
;; We turn the list into multiple values using (APPLY VALUES …).
(apply values
(map (lambda (file-path)
(let* ((path (string-append world-greeting-output "/" file-path))
(contents (call-with-input-file path get-string-all)))
(list path contents)))
;; SCANDIR from (ICE-9 FTW) returns the list of all files in a
;; directory (including ``.'' and ``..'', so we remove them with the
;; second argument, SELECT?, which specifies a predicate).
(scandir world-greeting-output
(lambda (path)
(not (or (string=? path ".")
(string=? path "..")))))))
⇒ ("/gnu/store/…-world-greeting/greetings" "Greetings, world!")
⇒ ("/gnu/store/…-world-greeting/hello" "Hello, world!")
⇒ ("/gnu/store/…-world-greeting/hi" "Hi, world!")</code></pre><p>Note that we define a procedure for building the output; we will need to build
more derivations in a very similar fashion later, so it helps to have this to
reuse instead of copying the code in <code>world-greeting-output</code>.</p><p>There are many other useful lowerable objects available as part of the gexp
library. These include <code>computed-file</code>, which accepts a gexp that builds
the output file, <code>program-file</code>, which creates an executable Scheme script in
the store using a g-expression, and <code>mixed-text-file</code>, which allows you to,
well, mix text and lowerable objects; it creates a file from the concatenation
of a sequence of strings and file-likes. The
<a href="https://guix.gnu.org/manual/en/html_node/G_002dExpressions.html">G-Expressions</a>
manual page has more details.</p><p>So, you may be wondering, at this point: there's so many lowerable objects
included with the g-expression library, surely there must be a way to define
more? Naturally, there is; this is Scheme, after all! We simply need to
acquaint ourselves with the <code>define-gexp-compiler</code> macro.</p><p>The most basic usage of <code>define-gexp-compiler</code> essentially creates a procedure
that takes as arguments a record to lower, the host system, and the target
system, and returns a derivation or store item as a monadic value in
<code>%store-monad</code>.</p><p>Let's try implementing a lowerable object representing a file that greets the
world. First, we'll define the record type:</p><pre><code class="language-scheme">(use-modules (srfi srfi-9))
(define-record-type <greeting-file>
(greeting-file greeting)
greeting?
(greeting greeting-file-greeting))</code></pre><p>Now we use <code>define-gexp-compiler</code> like so; note how we can use <code>lower-object</code>
to compile down any sort of lowerable object into the equivalent store item or
derivation; essentially, <code>lower-object</code> is just the procedure for applying the
right gexp-compiler to an object:</p><pre><code class="language-scheme">(use-modules (ice-9 i18n))
(define-gexp-compiler (greeting-file-compiler
(greeting-file <greeting-file>)
system target)
(lower-object
(let ((greeting (greeting-file-greeting greeting-file)))
(plain-file (string-append greeting "-greeting")
(string-append (string-locale-titlecase greeting) ", world!")))))</code></pre><p>Let's try it out now. Here's how we could rewrite our greetings directory
example from before using <code><greeting-file></code>:</p><pre><code class="language-scheme">(define world-greeting-2-output
(build-derivation
(gexp->derivation "world-greeting-2"
#~(begin
(mkdir #$output)
(symlink #$(greeting-file "hi")
(string-append #$output "/hi"))
(symlink #$(greeting-file "hello")
(string-append #$output "/hello"))
(symlink #$(greeting-file "greetings")
(string-append #$output "/greetings"))))))
(apply values
(map (lambda (file-path)
(let* ((path (string-append world-greeting-2-output
"/" file-path))
(contents (call-with-input-file path get-string-all)))
(list path contents)))
(scandir world-greeting-2-output
(lambda (path)
(not (or (string=? path ".")
(string=? path "..")))))))
⇒ ("/gnu/store/…-world-greeting-2/greetings" "Greetings, world!")
⇒ ("/gnu/store/…-world-greeting-2/hello" "Hello, world!")
⇒ ("/gnu/store/…-world-greeting-2/hi" "Hi, world!")</code></pre><p>Now, this is probably not worth a whole new gexp-compiler. How about something
a bit more complex? Sharp-eyed readers who are trying all this in the REPL may
have noticed the following output when they used <code>define-gexp-compiler</code>
(formatted for ease of reading):</p><pre><code class="language-scheme">⇒ #<<gexp-compiler>
type: #<record-type <greeting-file>>
lower: #<procedure … (greeting-file system target)>
expand: #<procedure default-expander (thing obj output)>></code></pre><p>Now, the purpose of <code>type</code> and <code>lower</code> is self-explanatory, but what's this
<code>expand</code> procedure here? Well, if you recall <code>file-append</code>, you may realise
that the text produced by a gexp-compiler for embedding into a g-expression
doesn't necessarily have to be the exact output path of the produced derivation.</p><p>There turns out to be another way to write a <code>define-gexp-compiler</code> form that
allows you to specify <em>both</em> the lowering procedure, which produces the
derivation or store item, and the expanding procedure, which produces the text.</p><p>Let's try making another new lowerable object; this one will let us build a
Guile package and expand to the path to its module directory. Here's our
record:</p><pre><code class="language-scheme">(define-record-type <module-directory>
(module-directory package)
module-directory?
(package module-directory-package))</code></pre><p>Here's how we define both a compiler and expander for our new record:</p><pre><code class="language-scheme">(use-modules (gnu packages guile)
(guix utils))
(define lookup-expander (@@ (guix gexp) lookup-expander))
(define-gexp-compiler module-directory-compiler <module-directory>
compiler => (lambda (obj system target)
(let ((package (module-directory-package obj)))
(lower-object package system #:target target)))
expander => (lambda (obj drv output)
(let* ((package (module-directory-package obj))
(expander (or (lookup-expander package)
(lookup-expander drv)))
(out (expander package drv output))
(guile (or (lookup-package-input package "guile")
guile-3.0))
(version (version-major+minor
(package-version guile))))
(string-append out "/share/guile/site/" version))))</code></pre><p>Let's try this out now:</p><pre><code class="language-scheme">(use-modules (gnu packages guile-xyz))
(define module-directory-output/guile-webutils
(build-derivation
(gexp->derivation "module-directory-output"
#~(symlink #$(module-directory guile-webutils) #$output))))
(readlink module-directory-output/guile-webutils)
⇒ "/gnu/store/…-guile-webutils-0.1-1.d309d65/share/guile/site/3.0"
(scandir module-directory-output/guile-webutils)
⇒ ("." ".." "webutils")
(define module-directory-output/guile2.2-webutils
(build-derivation
(gexp->derivation "module-directory-output"
#~(symlink #$(module-directory guile2.2-webutils) #$output))))
(readlink module-directory-output/guile2.2-webutils)
⇒ "/gnu/store/…-guile-webutils-0.1-1.d309d65/share/guile/site/2.2"
(scandir module-directory-output/guile2.2-webutils)
⇒ ("." ".." "webutils")</code></pre><p>Who knows why you'd want to do this, but it certainly works! We've looked at
why we need g-expressions, how they work, and how to extend them, and we've now
only got two more advanced features to cover: cross-build support, and modules.</p><h1>Importing External Modules</h1><p>Let's try using one of the helpful procedures from the <code>(guix build utils)</code>
module in a g-expression.</p><pre><code class="language-scheme">(define simple-directory-output
(build-derivation
(gexp->derivation "simple-directory"
#~(begin
(use-modules (guix build utils))
(mkdir-p (string-append #$output "/a/rather/simple/directory"))))))</code></pre><p>Looks fine, right? We've even got a <code>use-modules</code> in th--</p><pre><code class="language-Scheme">ERROR:
1. &store-protocol-error:
message: "build of `/gnu/store/…-simple-directory.drv' failed"
status: 100</code></pre><p>OUTRAGEOUS. Fortunately, there's an explanation to be found in the Guix build
log directory, <code>/var/log/guix/drvs</code>; locate the file using the first two
characters of the store hash as the subdirectory, and the rest as the file name,
and remember to use <code>zcat</code> or <code>zless</code>, as the logs are gzipped:</p><pre><code class="language-scheme">Backtrace:
9 (primitive-load "/gnu/store/…")
In ice-9/eval.scm:
721:20 8 (primitive-eval (begin (use-modules (guix build #)) (?)))
In ice-9/psyntax.scm:
1230:36 7 (expand-top-sequence ((begin (use-modules (guix ?)) #)) ?)
1090:25 6 (parse _ (("placeholder" placeholder)) ((top) #(# # ?)) ?)
1222:19 5 (parse _ (("placeholder" placeholder)) ((top) #(# # ?)) ?)
259:10 4 (parse _ (("placeholder" placeholder)) (()) _ c&e (eval) ?)
In ice-9/boot-9.scm:
3927:20 3 (process-use-modules _)
222:17 2 (map1 (((guix build utils))))
3928:31 1 (_ ((guix build utils)))
3329:6 0 (resolve-interface (guix build utils) #:select _ #:hide ?)
ice-9/boot-9.scm:3329:6: In procedure resolve-interface:
no code for module (guix build utils)</code></pre><p>It turns out <code>use-modules</code> can't actually find <code>(guix build utils)</code> at all.
There's no typo; it's just that to ensure the build is isolated, Guix builds
<code>module-import</code> and <code>module-importe-compiled</code> directories, and sets the
<em>Guile module path</em> within the build environment to contain said directories,
along with those containing the Guile standard library modules.</p><p>So, what to do? Turns out one of the fields in <code><gexp></code> is <code>modules</code>, which,
funnily enough, contains the names of the modules which will be used to build
the aforementioned directories. To add to this field, we use the
<code>with-imported-modules</code> macro. (<code>gexp->derivation</code> <em>does</em> provide a <code>modules</code>
parameter, but <code>with-imported-modules</code> lets you add the required modules
directly to the g-expression value, rather than later on.)</p><pre><code class="language-scheme">(define simple-directory-output
(build-derivation
(gexp->derivation "simple-directory"
(with-imported-modules '((guix build utils))
#~(begin
(use-modules (guix build utils))
(mkdir-p (string-append #$output "/a/rather/simple/directory")))))))
simple-directory-output
⇒ "/gnu/store/…-simple-directory"</code></pre><p>It works, yay. It's worth noting that while passing just the list of modules to
<code>with-imported-modules</code> works in this case, this is only because
<code>(guix build utils)</code> has no dependencies on other Guix modules. Were we to try
adding, say, <code>(guix build emacs-build-system)</code>, we'd need to use the
<code>source-module-closure</code> procedure to add its dependencies to the list:</p><pre><code class="language-scheme">(use-modules (guix modules))
(source-module-closure '((guix build emacs-build-system)))
⇒ ((guix build emacs-build-system)
(guix build gnu-build-system)
(guix build utils)
(guix build gremlin)
(guix elf)
(guix build emacs-utils))</code></pre><p>Here's another scenario: what if we want to use a module not from Guix or Guile
but a third-party library? In this example, we'll use <a href="https://github.com/aconchillo/guile-json">guile-json
</a>, a library for converting between
S-expressions and <a href="https://json.org">JavaScript Object Notation</a>.</p><p>We can't just <code>with-imported-modules</code> its modules, since it's not part of Guix,
so <code><gexp></code> provides another field for this purpose: <code>extensions</code>. Each of
these extensions is a lowerable object that produces a Guile package directory;
so usually a package. Let's try it out using the <code>guile-json-4</code> package to
produce a JSON file from a Scheme value within a g-expression.</p><pre><code class="language-scheme">(define helpful-guide-output
(build-derivation
(gexp->derivation "json-file"
(with-extensions (list guile-json-4)
#~(begin
(use-modules (json))
(mkdir #$output)
(call-with-output-file (string-append #$output "/helpful-guide.json")
(lambda (port)
(scm->json '((truth . "Guix is the best!")
(lies . "Guix isn't the best!"))
port))))))))
(call-with-input-file
(string-append helpful-guide-output "/helpful-guide.json")
get-string-all)
⇒ "{\"truth\":\"Guix is the best!\",\"lies\":\"Guix isn't the best!\"}"</code></pre><p>Amen to that, <code>helpful-guide.json</code>. Before we continue on to cross-compilation,
there's one last feature of <code>with-imported-modules</code> you should note. We can
add modules to a g-expression by name, but we can also create entirely new ones
using lowerable objects, such as in this pattern, which is used in several
places in the Guix source code to make an appropriately-configured
<code>(guix config)</code> module available:</p><pre><code class="language-scheme">(with-imported-modules `(((guix config) => ,(make-config.scm))
…)
…)</code></pre><p>In case you're wondering, <code>make-config.scm</code> is found in <code>(guix self)</code> and
returns a lowerable object that compiles to a version of the <code>(guix config)</code>
module, which contains constants usually substituted into the source code at
compile time.</p><h1>Native <code>ungexp</code></h1><p>There is another piece of syntax we can use with g-expressions, and it's called
<code>ungexp-native</code>. This helps us distinguish between native inputs and regular
host-built inputs in cross-compilation situations. We'll cover
cross-compilation in detail at a later date, but the gist of it is that it
allows you to compile a derivation for one architecture X, the target, using a
machine of architecture Y, the host, and Guix has excellent support for it.</p><p>If we cross-compile a g-expression G that <em>non-natively</em> <code>ungexp</code>s L1, a
lowerable object, from architecture Y to architecture X, both G and L1 will be
compiled for architecture X. However, if G <em>natively</em> <code>ungexp</code>s L1, G will be
compiled for X and L1 for Y.</p><p>Essentially, we use <code>ungexp-native</code> in situations where there would be no
difference between compiling on different architectures (for instance, if <code>L1</code>
were a <code>plain-file</code>), or where using L1 built for X would actually <em>break</em> G
(for instance, if <code>L1</code> corresponds to a compiled executable that needs to be run
during the build; the executable would fail to run on Y if it was built for X.)</p><p>The <code>ungexp-native</code> macro naturally has a corresponding reader syntax, <code>#+</code>, and
there's also <code>ungexp-native-splicing</code>, which is written as <code>#+@</code>. These two
pieces of syntax are used in the same way as their regular counterparts.</p><h1>Conclusion</h1><p>What have we learned in this post? To summarise:</p><ul><li>G-expressions are essentially abstractions on top of s-expressions used in
Guix to stage code, often for execution within a build environment or a
Shepherd service script.</li><li>Much like you can <code>unquote</code> external values within a <code>quasiquote</code> form, you
can <code>ungexp</code> external values to access them within a <code>gexp</code> form. The key
difference is that you may use not only s-expressions with <code>ungexp</code>, but other
g-expressions and lowerable objects too.</li><li>When a lowerable object is used with <code>ungexp</code>, the g-expression ultimately
receives the path to the object's store item (or whatever string the lowerable
object's expander produces), rather than the object itself.</li><li>A lowerable object is any record that has a "g-expression compiler" defined
for it using the <code>define-gexp-compiler</code> macro. G-expression compilers always
contain a <code>compiler</code> procedure, which converts an appropriate record into a
derivation, and sometimes an <code>expander</code> procedure, which produces the string
that is to be expanded to within g-expressions when the object is <code>ungexp</code>ed.</li><li>G-expressions record the list of modules available in their environment, which
you may expand using <code>with-imported-modules</code> to add Guix modules, and
<code>with-extensions</code> to add modules from third-party Guile packages.</li><li><code>ungexp-native</code> may be used within g-expressions to compile lowerable objects
for the host rather than the target system in cross-compilation scenarios.</li></ul><p>Mastering g-expressions is essential to understanding Guix's inner workings, so
the aim of this blog post is to be as thorough as possible. However, if you
still find yourself with questions, please don't hesitate to stop by at the IRC
channel <code>#guix:libera.chat</code> and mailing list <code>help-guix@gnu.org</code>; we'll be glad
to assist you!</p><p>Also note that due to the centrality of g-expressions to Guix, there exist a
plethora of alternative resources on this topic; here are some which you may
find useful:</p><ul><li>Arun Isaac's
<a href="https://www.systemreboot.net/post/deploy-scripts-using-g-expressions">post</a>
on using g-expressions with <code>guix deploy</code>.</li><li>Marius Bakke's
<a href="https://gexp.no/blog/guix-drops-part-3-g-expressions.html">"Guix Drops" post</a>
which explains g-expressions in a more "top-down" way.</li><li>This 2020
<a href="https://archive.fosdem.org/2020/schedule/event/gexpressionsguile/">FOSDEM talk</a>
by Christopher Marusich on the uses of g-expressions.</li><li>And, of course, the one and only original
<a href="https://hal.inria.fr/hal-01580582v1">g-expression paper</a> by Ludovic Courtès,
the original author of Guix.</li></ul><h4>About GNU Guix</h4><p><a href="https://guix.gnu.org">GNU Guix</a> is a transactional package manager and
an advanced distribution of the GNU system that <a href="https://www.gnu.org/distros/free-system-distribution-guidelines.html">respects user
freedom</a>.
Guix can be used on top of any system running the Hurd or the Linux
kernel, or it can be used as a standalone operating system distribution
for i686, x86_64, ARMv7, AArch64 and POWER9 machines.</p><p>In addition to standard package management features, Guix supports
transactional upgrades and roll-backs, unprivileged package management,
per-user profiles, and garbage collection. When used as a standalone
GNU/Linux distribution, Guix offers a declarative, stateless approach to
operating system configuration management. Guix is highly customizable
and hackable through <a href="https://www.gnu.org/software/guile">Guile</a>
programming interfaces and extensions to the
<a href="http://schemers.org">Scheme</a> language.</p>https://guix.gnu.org/blog/2023/dissecting-guix-part-2-the-store-monad//Dissecting Guix, Part 2: The Store Monad(2023-02-20T21:30:00Z2023-02-20T21:30:00Z Hello again! In the last post ,
we briefly mentioned the with-store and run-with-store macros. Today, we'll
be looking at those in further detail, along with the related monad library and
the %store-monad ! Typically, we use monads to chain operations together, and the %store-monad is
no different; it's used to combine operations that work on the Guix store (for
instance, creating derivations, building derivations, or adding data files to
the store). However, monads are a little hard to explain, and from a distance, they seem to
be quite incomprehensible. …<p>Hello again!</p><p>In <a href="https://guix.gnu.org/en/blog/2023/dissecting-guix-part-1-derivations/">the last post</a>,
we briefly mentioned the <code>with-store</code> and <code>run-with-store</code> macros. Today, we'll
be looking at those in further detail, along with the related monad library and
the <a href="https://guix.gnu.org/manual/devel/en/html_node/The-Store-Monad.html"><code>%store-monad</code></a>!</p><p>Typically, we use monads to chain operations together, and the <code>%store-monad</code> is
no different; it's used to combine operations that work on the Guix store (for
instance, creating derivations, building derivations, or adding data files to
the store).</p><p>However, monads are a little hard to explain, and from a distance, they seem to
be quite incomprehensible. So, I want you to erase them from your mind for now.
We'll come back to them later. And be aware that if you can't seem to get your
head around them, it's okay; you can understand most of the architecture of Guix
without understanding monads.</p><h1>Yes, No, Maybe So</h1><p>Let's instead implement another M of functional programming, <em><code>maybe</code></em> values,
representing a value that may or may not exist. For instance, there could be a
procedure that attempts to pop a stack, returning the result if there is one, or
<code>nothing</code> if the stack has no elements.</p><p><code>maybe</code> is a very common feature of statically-typed functional languages, and
you'll see it all over the place in Haskell and OCaml code. However, Guile is
dynamically typed, so we usually use ad-hoc <code>#f</code> values as the "null value"
instead of a proper "nothing" or "none".</p><p>Just for fun, though, we'll implement a proper <code>maybe</code> in Guile. Fire up that
REPL once again, and let's import a bunch of modules that we'll need:</p><pre><code class="language-scheme">(use-modules (ice-9 match)
(srfi srfi-9))</code></pre><p>We'll implement <code>maybe</code> as a record with two fields, <code>is?</code> and <code>value</code>. If the
value contains something, <code>is?</code> will be <code>#t</code> and <code>value</code> will contain the thing
in question, and if it's empty, <code>is?</code>'ll be <code>#f</code>.</p><pre><code class="language-scheme">(define-record-type <maybe>
(make-maybe is? value)
maybe?
(is? maybe-is?)
(value maybe-value))</code></pre><p>Now we'll define constructors for the two possible states:</p><pre><code class="language-scheme">(define (something value)
(make-maybe #t value))
(define (nothing)
(make-maybe #f #f)) ;the value here doesn't matter; we'll just use #f</code></pre><p>And make some silly functions that return optional values:</p><pre><code class="language-scheme">(define (remove-a str)
(if (eq? (string-ref str 0) #\a)
(something (substring str 1))
(nothing)))
(define (remove-b str)
(if (eq? (string-ref str 0) #\b)
(something (substring str 1))
(nothing)))
(remove-a "ahh")
⇒ #<<maybe> is?: #t value: "hh">
(remove-a "ooh")
⇒ #<<maybe> is?: #f value: #f>
(remove-b "bad")
⇒ #<<maybe> is?: #t value: "ad"></code></pre><p>But what if we want to compose the results of these functions?</p><h1>Keeping Your Composure</h1><p>As you might have guessed, this is not fun. Cosplaying as a compiler backend
typically isn't.</p><pre><code class="language-scheme">(let ((t1 (remove-a "abcd")))
(if (maybe-is? t1)
(remove-b (maybe-value t1))
(nothing)))
⇒ #<<maybe> is?: #t value: "cd">
(let ((t1 (remove-a "bbcd")))
(if (maybe-is? t1)
(remove-b (maybe-value t1))
(nothing)))
⇒ #<<maybe> is?: #f value: #f></code></pre><p>I can almost hear the heckling. Even worse, composing three:</p><pre><code class="language-scheme">(let* ((t1 (remove-a "abad"))
(t2 (if (maybe-is? t1)
(remove-b (maybe-value t1))
(nothing))))
(if (maybe-is? t2)
(remove-a (maybe-value t2))
(nothing)))
⇒ #<<maybe> is?: #t value: "d"></code></pre><p>So, how do we go about making this more bearable? Well, one way could be to
make <code>remove-a</code> and <code>remove-b</code> accept <code>maybe</code>s:</p><pre><code class="language-scheme">(define (remove-a ?str)
(match ?str
(($ <maybe> #t str)
(if (eq? (string-ref str 0) #\a)
(something (substring str 1))
(nothing)))
(_ (nothing))))
(define (remove-b ?str)
(match ?str
(($ <maybe> #t str)
(if (eq? (string-ref str 0) #\b)
(something (substring str 1))
(nothing)))
(_ (nothing))))</code></pre><p>Not at all pretty, but it works!</p><pre><code class="language-scheme">(remove-b (remove-a (something "abc")))
⇒ #<<maybe> is?: #t value: "c"></code></pre><p>Still, our procedures now require quite a bit of boilerplate. Might there be a
better way?</p><h1>The Ties That <code>>>=</code> Us</h1><p>First of all, we'll revert to our original definitions of <code>remove-a</code> and
<code>remove-b</code>, that is to say, the ones that take a regular value and return a
<code>maybe</code>.</p><pre><code class="language-scheme">(define (remove-a str)
(if (eq? (string-ref str 0) #\a)
(something (substring str 1))
(nothing)))
(define (remove-b str)
(if (eq? (string-ref str 0) #\b)
(something (substring str 1))
(nothing)))</code></pre><p>What if we tried introducing higher-order procedures (procedures that accept
other procedures as arguments) into the equation? Because we're functional
programmers and we have an unhealthy obsession with that sort of thing.</p><pre><code class="language-scheme">(define (maybe-chain maybe proc)
(if (maybe-is? maybe)
(proc (maybe-value maybe))
(nothing)))
(maybe-chain (something "abc")
remove-a)
⇒ #<<maybe> is?: #t value: "bc">
(maybe-chain (nothing)
remove-a)
⇒ #<<maybe> is?: #f value: #f></code></pre><p>It lives! To make it easier to compose procedures like this, we'll define a
macro that allows us to perform any number of sequenced operations with only one
composition form:</p><pre><code class="language-scheme">(define-syntax maybe-chain*
(syntax-rules ()
((_ maybe proc)
(maybe-chain maybe proc))
((_ maybe proc rest ...)
(maybe-chain* (maybe-chain maybe proc)
rest ...))))
(maybe-chain* (something "abad")
remove-a
remove-b
remove-a)
⇒ #<<maybe> is?: #t value: "d"></code></pre><p>Congratulations, you've just implemented the <code>bind</code> operation, commonly written
as <code>>>=</code>, for our <code>maybe</code> type. And it turns out that a monad is just any
container-like value for which <code>>>=</code> (along with another procedure called
<code>return</code>, which wraps a given value in the simplest possible form of a monad)
has been implemented.</p><p>A more formal definition would be that a monad is a mathematical object composed
of three parts: a type, a <code>bind</code> function, and a <code>return</code> function. So, how do
monads relate to Guix?</p><h1>New Wheel, Old Wheel</h1><p>Now that we've reinvented the wheel, we'd better learn to use the original
wheel. Guix provides a generic, high-level monads library, along with the two
generic monads <code>%identity-monad</code> and <code>%state-monad</code>, and the Guix-specific
<code>%store-monad</code>. Since <code>maybe</code> is not one of them, let's integrate our version
into the Guix monad system!</p><p>First we'll import the module that provides the aforementioned library:</p><pre><code class="language-scheme">(use-modules (guix monads))</code></pre><p>To define a monad's behaviour in Guix, we simply use the <code>define-monad</code> macro,
and provide two procedures: <code>bind</code>, and <code>return</code>.</p><pre><code class="language-scheme">(define-monad %maybe-monad
(bind maybe-chain)
(return something))</code></pre><p><code>bind</code> is just the procedure that we use to compose monadic procedure calls
together, and <code>return</code> is the procedure that wraps values in the most basic form
of the monad. A properly implemented <code>bind</code> and <code>return</code> must follow the
so-called <em>monad laws</em>:</p><ol><li><code>(bind (return x) proc)</code> must be equivalent to <code>(proc x)</code>.</li><li><code>(bind monad return)</code> must be equivalent to just <code>monad</code>.</li><li><code>(bind (bind monad proc-1) proc-2)</code> must be equivalent to
<code>(bind monad (lambda (x) (bind (proc-1 x) proc-2)))</code>.</li></ol><p>Let's verify that our <code>maybe-chain</code> and <code>something</code> procedures adhere to the
monad laws:</p><pre><code class="language-scheme">(define (mlaws-proc-1 x)
(something (+ x 1)))
(define (mlaws-proc-2 x)
(something (+ x 2)))
;; First law: the left identity.
(equal? (maybe-chain (something 0)
mlaws-proc-1)
(mlaws-proc-1 0))
⇒ #t
;; Second law: the right identity.
(equal? (maybe-chain (something 0)
something)
(something 0))
⇒ #t
;; Third law: associativity.
(equal? (maybe-chain (maybe-chain (something 0)
mlaws-proc-1)
mlaws-proc-2)
(maybe-chain (something 0)
(lambda (x)
(maybe-chain (mlaws-proc-1 x)
mlaws-proc-2))))
⇒ #t</code></pre><p>Now that we know they're valid, we can use the <code>with-monad</code> macro to tell Guix
to use these specific implementations of <code>bind</code> and <code>return</code>, and the <code>>>=</code>
macro to thread monads through procedure calls!</p><pre><code class="language-scheme">(with-monad %maybe-monad
(>>= (something "aabbc")
remove-a
remove-a
remove-b
remove-b))
⇒ #<<maybe> is?: #t value: "c"></code></pre><p>We can also now use <code>return</code>:</p><pre><code class="language-scheme">(with-monad %maybe-monad
(return 32))
⇒ #<<maybe> is?: #t value: 32></code></pre><p>But Guix provides many higher-level interfaces than <code>>>=</code> and <code>return</code>, as we
will see. There's <code>mbegin</code>, which evaluates monadic expressions without binding
them to symbols, returning the last one. This, however, isn't particularly
useful with our <code>%maybe-monad</code>, as it's only really usable if the monadic
operations within have side effects, just like the non-monadic <code>begin</code>.</p><p>There's also <code>mlet</code> and <code>mlet*</code>, which <em>do</em> bind the results of monadic
expressions to symbols, and are essentially equivalent to a chain of
<code>(>>= MEXPR (lambda (BINDING) ...))</code>:</p><pre><code class="language-scheme">;; This is equivalent...
(mlet* %maybe-monad ((str -> "abad") ;non-monadic binding uses the -> symbol
(str1 (remove-a str))
(str2 (remove-b str1)))
(remove-a str2))
⇒ #<<maybe> is?: #t value: "d">
;; ...to this:
(with-monad %maybe-monad
(>>= (return "abad")
(lambda (str)
(remove-a str))
(lambda (str1)
(remove-b str1))
(lambda (str2)
(remove-a str2))))</code></pre><p>Various abstractions over these two exist too, such as <code>mwhen</code> (a <code>when</code> plus an
<code>mbegin</code>), <code>munless</code> (an <code>unless</code> plus an <code>mbegin</code>), and <code>mparameterize</code>
(dynamically-scoped value rebinding, like <code>parameterize</code>, in a monadic context).
<code>lift</code> takes a procedure and a monad and creates a new procedure that returns
a monadic value.</p><p>There are also interfaces for manipulating lists wrapped in monads; <code>listm</code>
creates such a list, <code>sequence</code> turns a list of monads into a list wrapped in a
monad, and the <code>anym</code>, <code>mapm</code>, and <code>foldm</code> procedures are like their non-monadic
equivalents, except that they return lists wrapped in monads.</p><p>This is all well and good, you may be thinking, but why does Guix need a monad
library, anyway? The answer is technically that it doesn't. But building on
the monad API makes a lot of things much easier, and to learn why, we're going
to look at one of Guix's built-in monads.</p><h1>In a State</h1><p>Guix implements a monad called <code>%state-monad</code>, and it works with single-argument
procedures returning two values. Behold:</p><pre><code class="language-scheme">(with-monad %state-monad
(return 33))
⇒ #<procedure 21dc9a0 at <unknown port>:1106:22 (state)></code></pre><p>The <code>run-with-state</code> value turns this procedure into an actually useful value,
or, rather, two values:</p><pre><code class="language-scheme">(run-with-state (with-monad %state-monad (return 33))
(list "foo" "bar" "baz"))
⇒ 33
⇒ ("foo" "bar" "baz")</code></pre><p>What can this actually do for us, though? Well, it gets interesting if we do
some <code>>>=</code>ing:</p><pre><code class="language-scheme">(define state-seq
(mlet* %state-monad ((number (return 33)))
(state-push number)))
result
⇒ #<procedure 7fcb6f466960 at <unknown port>:1484:24 (state)>
(run-with-state state-seq (list 32))
⇒ (32)
⇒ (33 32)
(run-with-state state-seq (list 30 99))
⇒ (30 99)
⇒ (33 30 99)</code></pre><p>What is <code>state-push</code>? It's a monadic procedure for <code>%state-monad</code> that takes
whatever's currently in the first value (the primary value) and pushes it onto
the second value (the state value), which is assumed to be a list, returning the
old state value as the primary value and the new list as the state value.</p><p>So, when we do <code>(run-with-state result (list 32))</code>, we're passing <code>(list 32)</code> as
the initial state value, and then the <code>>>=</code> form passes that and <code>33</code> to
<code>state-push</code>. What <code>%state-monad</code> allows us to do is thread together some
procedures that require some kind of state, while essentially pretending the
state value is stored globally, like you might do in, say, C, and then retrieve
both the final state and the result at the end!</p><p>If you're a bit confused, don't worry. We'll write some of our own
<code>%state-monad</code>-based monadic procedures and hopefully all will become clear.
Consider, for instance, the
<a href="https://en.wikipedia.org/wiki/Fibonacci_number">Fibonacci sequence</a>, in which
each value is computed by adding the previous two. We could use the
<code>%state-monad</code> to compute Fibonacci numbers by storing the previous number as
the primary value and the number before that as the state value:</p><pre><code class="language-scheme">(define (fibonacci-thing value)
(lambda (state)
(values (+ value state)
value)))</code></pre><p>Now we can feed our Fibonacci-generating procedure the first value using
<code>run-with-state</code> and the second using <code>return</code>:</p><pre><code class="language-scheme">(run-with-state
(mlet* %state-monad ((starting (return 1))
(n1 (fibonacci-thing starting))
(n2 (fibonacci-thing n1)))
(fibonacci-thing n2))
0)
⇒ 3
⇒ 2
(run-with-state
(mlet* %state-monad ((starting (return 1))
(n1 (fibonacci-thing starting))
(n2 (fibonacci-thing n1))
(n3 (fibonacci-thing n2))
(n4 (fibonacci-thing n3))
(n5 (fibonacci-thing n4)))
(fibonacci-thing n5))
0)
⇒ 13
⇒ 8</code></pre><p>This is all very nifty, and possibly useful in general, but what does this have
to do with Guix? Well, many Guix store-based operations are meant to be used
in concert with yet another monad, called the <code>%store-monad</code>. But if we look at
<code>(guix store)</code>, where <code>%store-monad</code> is defined...</p><pre><code class="language-scheme">(define-alias %store-monad %state-monad)
(define-alias store-return state-return)
(define-alias store-bind state-bind)</code></pre><p>It was all a shallow façade! All the "store monad" is is a special case of the
state monad, where a value representing the store is passed as the state value.</p><h1>Lies, Damned Lies, and Abstractions</h1><p>We mentioned that, technically, we didn't need monads for Guix. Indeed, many
(now deprecated) procedures take a store value as the argument, such as
<code>build-expression->derivation</code>. However, monads are far more elegant and
simplify store code by quite a bit.</p><p><code>build-expression->derivation</code>, being deprecated, should never of course be
used. For one thing, it uses the "quoted build expression" style, rather than
G-expressions (we'll discuss gexps another time). The best way to create a
derivation from some basic build code is to use the new-fangled
<code>gexp->derivation</code> procedure:</p><pre><code class="language-scheme">(use-modules (guix gexp)
(gnu packages irc))
(define symlink-irssi
(gexp->derivation "link-to-irssi"
#~(symlink #$(file-append irssi "/bin/irssi") #$output)))
⇒ #<procedure 7fddcc7b81e0 at guix/gexp.scm:1180:2 (state)></code></pre><p>You don't have to understand the <code>#~(...)</code> form yet, only everything surrounding
it. We can see that this <code>gexp->derivation</code> returns a procedure taking the
initial state (store), just like our <code>%state-monad</code> procedures did, and like we
used <code>run-with-state</code> to pass the initial state to a <code>%state-monad</code> monadic
value, we use our old friend <code>run-with-store</code> when we have a <code>%store-monad</code>
monadic value!</p><pre><code class="language-scheme">(define symlink-irssi-drv
(with-store store
(run-with-store store
symlink-irssi)))
⇒ #<derivation /gnu/store/q7kwwl4z6psifnv4di1p1kpvlx06fmyq-link-to-irssi.drv => /gnu/store/6a94niigx4ii0ldjdy33wx9anhifr25x-link-to-irssi 7fddb7ef52d0></code></pre><p>Let's just check this derivation is as expected by reading the code from the
builder script.</p><pre><code class="language-scheme">(define symlink-irssi-builder
(list-ref (derivation-builder-arguments symlink-irssi-drv) 1))
(call-with-input-file symlink-irssi-builder
(lambda (port)
(read port)))
⇒ (symlink
"/gnu/store/hrlmypx1lrdjlxpkqy88bfrzg5p0bn6d-irssi-1.4.3/bin/irssi"
((@ (guile) getenv) "out"))</code></pre><p>And indeed, it symlinks the <code>irssi</code> binary to the output path. Some other,
higher-level, monadic procedures include <code>interned-file</code>, which copies a file
from outside the store into it, and <code>text-file</code>, which copies some text into it.
Generally, these procedures aren't used, as there are higher-level procedures
that perform similar functions (which we will discuss later), but for the sake
of this blog post, here's an example:</p><pre><code class="language-scheme">(with-store store
(run-with-store store
(text-file "unmatched-paren"
"( <paren@disroot.org>")))
⇒ "/gnu/store/v6smacxvdk4yvaa3s3wmd54lixn1dp3y-unmatched-paren"</code></pre><h1>Conclusion</h1><p>What have we learned about monads? The key points we can take away are:</p><ol><li>Monads are a way of composing together procedures and values that are wrapped
in containers that give them extra context, like <code>maybe</code> values.</li><li>Guix provides a high-level monad library that compensates for Guile's lack of
static typing or an interface-like system.</li><li>The <code>(guix monads)</code> module provides the state monad, which allows you to
thread state through procedures, allowing you to essentially pretend it's a
global variable that's modified by each procedure.</li><li>Guix uses the store monad frequently to thread a store connection through
procedures that need it.</li><li>The store monad is really just the state monad in disguise, where the state
value is used to thread the store object through monadic procedures.</li></ol><p>If you've read this post in its entirety but still don't yet quite get it, don't
worry. Try to modify and tinker about with the examples, and ask any questions
on the IRC channel <code>#guix:libera.chat</code> and mailing list at <code>help-guix@gnu.org</code>,
and hopefully it will all click eventually!</p><h4>About GNU Guix</h4><p><a href="https://guix.gnu.org">GNU Guix</a> is a transactional package manager and
an advanced distribution of the GNU system that <a href="https://www.gnu.org/distros/free-system-distribution-guidelines.html">respects user
freedom</a>.
Guix can be used on top of any system running the Hurd or the Linux
kernel, or it can be used as a standalone operating system distribution
for i686, x86_64, ARMv7, AArch64 and POWER9 machines.</p><p>In addition to standard package management features, Guix supports
transactional upgrades and roll-backs, unprivileged package management,
per-user profiles, and garbage collection. When used as a standalone
GNU/Linux distribution, Guix offers a declarative, stateless approach to
operating system configuration management. Guix is highly customizable
and hackable through <a href="https://www.gnu.org/software/guile">Guile</a>
programming interfaces and extensions to the
<a href="http://schemers.org">Scheme</a> language.</p>https://guix.gnu.org/blog/2023/dissecting-guix-part-1-derivations//Dissecting Guix, Part 1: Derivations(2023-01-04T12:30:00Z2023-01-04T12:30:00Z To a new user, Guix's functional architecture can seem quite alien, and possibly
offputting. With a combination of extensive #guix -querying, determined
manual-reading, and plenty of source-perusing, they may eventually figure out
how everything fits together by themselves, but this can be frustrating and
often takes a fairly long time. However, once you peel back the layers, the "Nix way" is actually rather
elegant, if perhaps not as simple as the mutable, imperative style implemented
by the likes of dpkg and
pacman .
This series of blog posts will cover basic Guix concepts, taking a "ground-up"
approach…<p>To a new user, Guix's functional architecture can seem quite alien, and possibly
offputting. With a combination of extensive <code>#guix</code>-querying, determined
manual-reading, and plenty of source-perusing, they may eventually figure out
how everything fits together by themselves, but this can be frustrating and
often takes a fairly long time.</p><p>However, once you peel back the layers, the "Nix way" is actually rather
elegant, if perhaps not as simple as the mutable, imperative style implemented
by the likes of <a href="https://wiki.debian.org/dpkg"><code>dpkg</code></a> and
<a href="https://wiki.archlinux.org/title/pacman"><code>pacman</code></a>.
This series of blog posts will cover basic Guix concepts, taking a "ground-up"
approach by dealing with lower-level concepts first, and hopefully make those
months of information-gathering unnecessary.</p><p>Before we dig in to Guix-specific concepts, we'll need to learn about one
inherited from <a href="https://nixos.org">Nix</a>, the original functional package manager
and the inspiration for Guix; the idea of a
<a href="https://guix.gnu.org/manual/devel/en/html_node/Derivations.html"><em>derivation</em></a>
and its corresponding <em>store items</em>.</p><p>These concepts were originally described by Eelco Dolstra, the original author
of Nix, in their <a href="https://edolstra.github.io/pubs/phd-thesis.pdf">PhD thesis</a>;
see <em>§ 2.1 The Nix store</em> and <em>§ 2.4 Store Derivations</em>.</p><h1>Store Items</h1><p>As you probably know, everything that Guix builds is stored in the <em>store</em>,
which is almost always the <code>/gnu/store</code> directory. It's the job of the
<a href="https://guix.gnu.org/manual/en/html_node/Invoking-guix_002ddaemon.html"><code>guix-daemon</code></a>
to manage the store and build things. If you run
<a href="https://guix.gnu.org/manual/en/html_node/Invoking-guix-build.html"><code>guix build PKG</code></a>,
<code>PKG</code> will be built or downloaded from a substitute server if available, and a
path to an item in the store will be displayed.</p><pre><code>$ guix build irssi
/gnu/store/v5pd69j3hjs1fck4b5p9hd91wc8yf5qx-irssi-1.4.3</code></pre><p>This item contains the final result of building <a href="https://irssi.org"><code>irssi</code></a>.
Let's peek inside:</p><pre><code>$ ls $(guix build irssi)
bin/ etc/ include/ lib/ share/
$ ls $(guix build irssi)/bin
irssi*</code></pre><p><code>irssi</code> is quite a simple package. What about a more complex one, like
<a href="https://docs.gtk.org/glib"><code>glib</code></a>?</p><pre><code>$ guix build glib
/gnu/store/bx8qq76idlmjrlqf1faslsq6zjc6f426-glib-2.73.3-bin
/gnu/store/j65bhqwr7qq7l77nj0ahmk1f1ilnjr3a-glib-2.73.3-debug
/gnu/store/3pn4ll6qakgfvfpc4mw89qrrbsgj3jf3-glib-2.73.3-doc
/gnu/store/dvsk6x7d26nmwsqhnzws4iirb6dhhr1d-glib-2.73.3
/gnu/store/4c8ycz501n2d0xdi4blahvnbjhd5hpa8-glib-2.73.3-static</code></pre><p><code>glib</code> produces five <code>/gnu/store</code> items, because it's possible for a package to
produce multiple <a href="https://guix.gnu.org/manual/en/html_node/Packages-with-Multiple-Outputs.html">outputs</a>.
Each output can be referred to separately, by suffixing a package's name with
<code>:OUTPUT</code> where supported. For example, this
<a href="https://guix.gnu.org/manual/en/html_node/Invoking-guix-package.html"><code>guix install</code></a>
invocation will add <code>glib</code>'s <code>bin</code> output to your profile:</p><pre><code>$ guix install glib:bin</code></pre><p>The default output is <code>out</code>, so when you pass <code>glib</code> by itself to that command,
it will actually install <code>glib:out</code> to the profile.</p><p><code>guix build</code> also provides the <code>--source</code> flag, which produces the store item
corresponding to the given package's downloaded source code.</p><pre><code>$ guix build --source irssi
/gnu/store/cflbi4nbak0v9xbyc43lamzl4a539hhb-irssi-1.4.3.tar.xz
$ guix build --source glib
/gnu/store/d22wzjq3xm3q8hwnhbgk2xd3ph7lb6ay-glib-2.73.3.tar.xz</code></pre><p>But how does Guix know how to build these store outputs in the first place?
That's where derivations come in.</p><h1><code>.drv</code> Files</h1><p>You've probably seen these being printed by the Guix program now and again.
Derivations, represented in the daemon's eyes by <code>.drv</code> files, contain
instructions for building store items. We can retrieve the paths of these
<code>.drv</code> files with the <code>guix build --derivations</code> command:</p><pre><code>$ guix build --derivations irssi
/gnu/store/zcgmhac8r4kdj2s6bcvcmhh4k35qvihx-irssi-1.4.3.drv</code></pre><p><code>guix build</code> can actually also accept derivation paths as an argument, in lieu
of a package, like so:</p><pre><code>$ guix build /gnu/store/zcgmhac8r4kdj2s6bcvcmhh4k35qvihx-irssi-1.4.3.drv
/gnu/store/v5pd69j3hjs1fck4b5p9hd91wc8yf5qx-irssi-1.4.3</code></pre><p>Let's look inside this derivation file.</p><pre><code>Derive([("out","/gnu/store/v5pd69j3hjs1fck4b5p9hd91wc8yf5qx-irssi-1.4.3","","")],[("/gnu/store/9mv9xg4kyj4h1cvsgrw7b9x34y8yppph-glib-2.70.2.drv",["out"]),("/gnu/store/baqpbl4wck7nkxrbyc9nlhma7kq5dyfl-guile-2.0.14.drv",["out"]),("/gnu/store/bfirgq65ndhf63nn4q6vlkbha9zd931q-openssl-1.1.1l.drv",["out"]),("/gnu/store/gjwpqzvfhz13shix6a6cs2hjc18pj7wy-module-import-compiled.drv",["out"]),("/gnu/store/ij8651x4yh53hhcn6qw2644nhh2s8kcn-glib-2.70.2.drv",["out"]),("/gnu/store/jg2vv6yc2yqzi3qzs82dxvqmi5k21lhy-irssi-1.4.3.drv",["out"]),("/gnu/store/qggpjl9g6ic3cq09qrwkm0dfsdjf7pyr-glibc-utf8-locales-2.33.drv",["out"]),("/gnu/store/zafabw13yyhz93jwrcz7axak1kn1f2cx-openssl-1.1.1s.drv",["out"])],["/gnu/store/af18nrrsk98c5a71h3fifnxg1zi5mx7y-module-import","/gnu/store/qnrwmby5cwqdqxyiv1ga6azvakmdvgl7-irssi-1.4.3-builder"],"x86_64-linux","/gnu/store/hnr4r2d0h0xarx52i6jq9gvsrlc3q81a-guile-2.0.14/bin/guile",["--no-auto-compile","-L","/gnu/store/af18nrrsk98c5a71h3fifnxg1zi5mx7y-module-import","-C","/gnu/store/6rkkvvb7pl1l9ng8vvywvwf357vhm3va-module-import-compiled","/gnu/store/qnrwmby5cwqdqxyiv1ga6azvakmdvgl7-irssi-1.4.3-builder"],[("allowSubstitutes","0"),("guix properties","((type . graft) (graft (count . 2)))"),("out","/gnu/store/v5pd69j3hjs1fck4b5p9hd91wc8yf5qx-irssi-1.4.3"),("preferLocalBuild","1")])</code></pre><p>It's... not exactly human-readable. We could try to format it and break it
down, but it'd still be pretty hard to understand, since <code>.drv</code> files contain no
labels for the fields or any other human-readable indicators. Instead, we're
going to explore derivations in a Guile REPL.</p><h1>Exploring Guix Interactively</h1><p>Before we continue, we'll want to start a REPL, so that we can try out the Guix
Guile API interactively. To run a REPL in the terminal, simply
<a href="https://guix.gnu.org/manual/devel/en/html_node/Using-Guix-Interactively.html">call <code>guix repl</code></a>.</p><p>If you're using Emacs, you can instead install
<a href="https://nongnu.org/geiser">Geiser</a>, which provides a comfortable Emacs UI for
various Lisp REPLs, invoke <code>guix repl --listen=tcp:37146 &</code>, and type
<code>M-x geiser-connect RET RET RET</code> to connect to the running Guile instance.</p><p>Your <code>.guile</code> file may contain code for enabling colours and readline bindings
that Geiser will choke on. The default Guix System <code>.guile</code> contains code to
suppress these features when <code>INSIDE_EMACS</code> is set, so you'll need to run
<code>guix repl</code> like this:</p><pre><code class="language-sh">INSIDE_EMACS=1 guix repl --listen=tcp:37146 &</code></pre><p>There are a few Guix modules we'll need. Run this Scheme code to import them:</p><pre><code class="language-scheme">(use-modules (guix)
(guix derivations)
(guix gexp)
(guix packages)
(guix store)
(gnu packages glib)
(gnu packages irc))</code></pre><p>We now have access to the store, G-expression, package, and derivation APIs,
along with the <code>irssi</code> and <code>glib</code> <code><package></code> objects.</p><h1>Creating a <code><derivation></code></h1><p>The Guix API for derivations revolves around the <code><derivation></code> record, which is
the Scheme representation of that whole block of text surrounded by
<code>Derive(...)</code>. If we look in <code>guix/derivations.scm</code>, we can see that it's
defined like this:</p><pre><code class="language-scheme">(define-immutable-record-type <derivation>
(make-derivation outputs inputs sources system builder args env-vars
file-name)
derivation?
(outputs derivation-outputs) ; list of name/<derivation-output> pairs
(inputs derivation-inputs) ; list of <derivation-input>
(sources derivation-sources) ; list of store paths
(system derivation-system) ; string
(builder derivation-builder) ; store path
(args derivation-builder-arguments) ; list of strings
(env-vars derivation-builder-environment-vars) ; list of name/value pairs
(file-name derivation-file-name)) ; the .drv file name</code></pre><p>With the exception of <code>file-name</code>, each of those fields corresponds to a field
in the <code>Derive(...)</code> form. Before we can examine them, though, we need to
figure out how to <em>lower</em> that <code>irssi</code> <code><package></code> object into a derivation.</p><p><code>guix repl</code> provides the <code>,lower</code> command to create derivations quickly,
as shown in this sample REPL session:</p><pre><code>scheme@(guile-user)> ,use (guix)
scheme@(guile-user)> ,use (gnu packages irc)
scheme@(guile-user)> irssi
$1 = #<package irssi@1.4.3 gnu/packages/irc.scm:153 7f3ff98e0c60>
scheme@(guile-user)> ,lower irssi
$2 = #<derivation /gnu/store/drjfddvlblpr635jazrg9kn5azd9hsbj-irssi-1.4.3.drv => /gnu/store/v5pd69j3hjs1fck4b5p9hd91wc8yf5qx-irssi-1.4.3 7f3ff7782d70>
;; Below we use the $N variable automatically bound by the REPL.
scheme@(guile-user)> (derivation-system $2)
$3 = "x86_64-linux"</code></pre><p>Since <code>,lower</code> is a REPL command, however, we can't use it in proper Scheme
code. It's quite useful for exploring specific derivations interactively, but
since the purpose of this blog post is to explain how things work inside, we're
going to use the pure-Scheme approach here.</p><p>The procedure we need to use to turn a high-level object like <code><package></code> into a
derivation is called <code>lower-object</code>; more on that in a future post. However,
this doesn't initially produce a derivation:</p><pre><code class="language-scheme">(pk (lower-object irssi))
;;; (#<procedure 7fe17c7af540 at guix/store.scm:1994:2 (state)>)</code></pre><p><code>pk</code> is an abbreviation for the procedure <code>peek</code>, which takes the given object,
writes a representation of it to the output, and returns it. It's especially
handy when you want to view an intermediate value in a complex expression.</p><p>The returned object is a monadic value (more on those in the next post on
monads) that needs to be evaluated in the context of a store connection. We do
this by first using <code>with-store</code> to connect to the store and bind the connection
to a name, then wrapping the <code>lower-object</code> call with <code>run-with-store</code>:</p><pre><code class="language-scheme">(define irssi-drv
(pk (with-store %store
(run-with-store %store
(lower-object irssi)))))
;;; (#<derivation /gnu/store/zcgmhac8r4kdj2s6bcvcmhh4k35qvihx-irssi-1.4.3.drv => /gnu/store/v5pd69j3hjs1fck4b5p9hd91wc8yf5qx-irssi-1.4.3 7fe1902b6140>)
(define glib-drv
(pk (with-store %store
(run-with-store %store
(lower-object glib)))))
;;; (#<derivation /gnu/store/81qqs7xah2ln39znrji4r6xj85zi15bi-glib-2.70.2.drv => /gnu/store/lp7k9ygvpwxgxjvmf8bix8d2aar0azr7-glib-2.70.2-bin /gnu/store/22mkp8cr6rxg6w8br9q8dbymf51b44m8-glib-2.70.2-debug /gnu/store/a6qb5arvir4vm1zlkp4chnl7d8qzzd7x-glib-2.70.2 /gnu/store/y4ak268dcdwkc6lmqfk9g1dgk2jr9i34-glib-2.70.2-static 7fe17ca13b90>)</code></pre><p>And we have liftoff! Now we've got two <code><derivation></code> records to play with.</p><h1>Exploring <code><derivation></code></h1><h2><code><derivation-output></code></h2><p>The first "argument" in the <code>.drv</code> file is <code>outputs</code>, which tells the Guix
daemon about the outputs that this build can produce:</p><pre><code class="language-scheme">(define irssi-outputs
(pk (derivation-outputs irssi-drv)))
;;; ((("out" . #<<derivation-output> path: "/gnu/store/v5pd69j3hjs1fck4b5p9hd91wc8yf5qx-irssi-1.4.3" hash-algo: #f hash: #f recursive?: #f>)))
(pk (assoc-ref irssi-outputs "out"))
(define glib-outputs
(pk (derivation-outputs glib-drv)))
;;; ((("bin" . #<<derivation-output> path: "/gnu/store/lp7k9ygvpwxgxjvmf8bix8d2aar0azr7-glib-2.70.2-bin" hash-algo: #f hash: #f recursive?: #f>) ("debug" . #<<derivation-output> path: "/gnu/store/22mkp8cr6rxg6w8br9q8dbymf51b44m8-glib-2.70.2-debug" hash-algo: #f hash: #f recursive?: #f>) ("out" . #<<derivation-output> path: "/gnu/store/a6qb5arvir4vm1zlkp4chnl7d8qzzd7x-glib-2.70.2" hash-algo: #f hash: #f recursive?: #f>) ("static" . #<<derivation-output> path: "/gnu/store/y4ak268dcdwkc6lmqfk9g1dgk2jr9i34-glib-2.70.2-static" hash-algo: #f hash: #f recursive?: #f>)))
(pk (assoc-ref glib-outputs "bin"))
;;; (#<<derivation-output> path: "/gnu/store/lp7k9ygvpwxgxjvmf8bix8d2aar0azr7-glib-2.70.2-bin" hash-algo: #f hash: #f recursive?: #f>)</code></pre><p>It's a simple association list mapping output names to <code><derivation-output></code>
records, and it's equivalent to the first "argument" in the <code>.drv</code> file:</p><pre><code>[ ("out", "/gnu/store/v5pd69j3hjs1fck4b5p9hd91wc8yf5qx-irssi-1.4.3", "", "")
]</code></pre><p>The <code>hash-algo</code> and <code>hash</code> fields are for storing the content hash and the
algorithm used with that hash for what we term a <em>fixed-output derivation</em>,
which is essentially a derivation where we know what the hash of the content
will be in advance. For instance, <code>origin</code>s produce fixed-output derivations:</p><pre><code class="language-scheme">(define irssi-src-drv
(pk (with-store %store
(run-with-store %store
(lower-object (package-source irssi))))))
;;; (#<derivation /gnu/store/mcz3vzq7lwwaqjb8dy7cd69lvmi6d241-irssi-1.4.3.tar.xz.drv => /gnu/store/cflbi4nbak0v9xbyc43lamzl4a539hhb-irssi-1.4.3.tar.xz 7fe17b3c8d70>)
(define irssi-src-outputs
(pk (derivation-outputs irssi-src-drv)))
;;; ((("out" . #<<derivation-output> path: "/gnu/store/cflbi4nbak0v9xbyc43lamzl4a539hhb-irssi-1.4.3.tar.xz" hash-algo: sha256 hash: #vu8(185 63 113 82 35 163 34 230 127 66 182 26 8 165 18 174 41 227 75 212 165 61 127 34 55 102 102 10 170 90 4 52) recursive?: #f>)))
(pk (assoc-ref irssi-src-outputs "out"))
;;; (#<<derivation-output> path: "/gnu/store/cflbi4nbak0v9xbyc43lamzl4a539hhb-irssi-1.4.3.tar.xz" hash-algo: sha256 hash: #vu8(185 63 113 82 35 163 34 230 127 66 182 26 8 165 18 174 41 227 75 212 165 61 127 34 55 102 102 10 170 90 4 52) recursive?: #f>)</code></pre><p>Note how the <code>hash</code> and <code>hash-algo</code> now have values.</p><p>Perceptive readers may note that the <code><derivation-output></code> has four fields,
whereas the tuple in the <code>.drv</code> file only has three (minus the label). The
serialisation of <code>recursive?</code> is done by adding the prefix <code>r:</code> to the
<code>hash-algo</code> field, though its actual purpose is difficult to explain, and is out
of scope for this post.</p><h2><code><derivation-input></code></h2><p>The next field is <code>inputs</code>, which corresponds to the second field in the <code>.drv</code>
file format:</p><pre><code>[ ("/gnu/store/9mv9xg4kyj4h1cvsgrw7b9x34y8yppph-glib-2.70.2.drv", ["out"]),
("/gnu/store/baqpbl4wck7nkxrbyc9nlhma7kq5dyfl-guile-2.0.14.drv", ["out"]),
("/gnu/store/bfirgq65ndhf63nn4q6vlkbha9zd931q-openssl-1.1.1l.drv", ["out"]),
("/gnu/store/gjwpqzvfhz13shix6a6cs2hjc18pj7wy-module-import-compiled.drv", ["out"]),
("/gnu/store/ij8651x4yh53hhcn6qw2644nhh2s8kcn-glib-2.70.2.drv", ["out"]),
("/gnu/store/jg2vv6yc2yqzi3qzs82dxvqmi5k21lhy-irssi-1.4.3.drv", ["out"]),
("/gnu/store/qggpjl9g6ic3cq09qrwkm0dfsdjf7pyr-glibc-utf8-locales-2.33.drv", ["out"]),
("/gnu/store/zafabw13yyhz93jwrcz7axak1kn1f2cx-openssl-1.1.1s.drv", ["out"])
]</code></pre><p>Here, each tuple specifies a derivation that needs to be built before this
derivation can be built, and the outputs of the derivation that the build
process of this derivation uses. Let's grab us the Scheme equivalent:</p><pre><code class="language-scheme">(define irssi-inputs
(pk (derivation-inputs irssi-drv)))
;;; [a fairly large amount of output]
(pk (car irssi-inputs))
;;; (#<<derivation-input> drv: #<derivation /gnu/store/9mv9xg4kyj4h1cvsgrw7b9x34y8yppph-glib-2.70.2.drv => /gnu/store/2jj2mxn6wfrcw7i85nywk71mmqbnhzps-glib-2.70.2 7fe1902b6640> sub-derivations: ("out")>)</code></pre><p>Unlike <code>derivation-outputs</code>, <code>derivation-inputs</code> maps 1:1 to the <code>.drv</code>
form; the <code>drv</code> field is a <code><derivation></code> to be built, and the
<code>sub-derivations</code> field is a list of outputs.</p><h2>Builder Configuration</h2><p>The other fields are simpler; none of them involve new records. The third is
<code>derivation-sources</code>, which contains a list of all store items used in the build
which aren't themselves built using derivations, whereas <code>derivation-inputs</code>
contains the dependencies which are.</p><p>This list usually just contains the path to the Guile <em>build script</em> that
realises the store items when run, which we'll examine in a later post, and
the path to a directory containing extra modules to add to the build script's
<code>%load-path</code>, called <code>/gnu/store/...-module-import</code>.</p><p>The next field is <code>derivation-system</code>, which specifies the system type (such as
<code>x86_64-linux</code>) we're building for. Then we have <code>derivation-builder</code>, pointing
to the <code>guile</code> executable that runs the build script; and the second-to-last is
<code>derivation-builder-arguments</code>, which is a list of arguments to pass to
<code>derivation-builder</code>. Note how we use <code>-L</code> and <code>-C</code> to extend the Guile
<code>%load-path</code> and <code>%load-compiled-path</code> to include the <code>module-import</code> and
<code>module-import-compiled</code> directories:</p><pre><code class="language-scheme">(pk (derivation-system irssi-drv))
;;; ("x86_64-linux")
(pk (derivation-builder irrsi-drv))
;;; ("/gnu/store/hnr4r2d0h0xarx52i6jq9gvsrlc3q81a-guile-2.0.14/bin/guile")
(pk (derivation-builder-arguments irrsi-drv))
;;; (("--no-auto-compile" "-L" "/gnu/store/af18nrrsk98c5a71h3fifnxg1zi5mx7y-module-import" "-C" "/gnu/store/6rkkvvb7pl1l9ng8vvywvwf357vhm3va-module-import-compiled" "/gnu/store/qnrwmby5cwqdqxyiv1ga6azvakmdvgl7-irssi-1.4.3-builder"))</code></pre><p>The final field contains a list of environment variables to set before we start
the build process:</p><pre><code class="language-scheme">(pk (derivation-builder-environment-vars irssi-drv))
;;; ((("allowSubstitutes" . "0") ("guix properties" . "((type . graft) (graft (count . 2)))") ("out" . "/gnu/store/v5pd69j3hjs1fck4b5p9hd91wc8yf5qx-irssi-1.4.3") ("preferLocalBuild" . "1")))</code></pre><p>The last record field, <code>derivation-file-name</code> contains the path to the <code>.drv</code>
file, and so isn't represented in a serialised derivation.</p><h1>Utilising <code><derivation></code></h1><p>Speaking of serialisation, to convert between the <code>.drv</code> text format and the
Scheme <code><derivation></code> record, you can use <code>write-derivation</code>, <code>read-derivation</code>,
and <code>read-derivation-from-file</code>:</p><pre><code class="language-scheme">(define manual-drv
(with-store %store
(derivation %store "manual"
"/bin/sh" '())))
(write-derivation manual-drv (current-output-port))
;;; -| Derive([("out","/gnu/store/kh7fais2zab22fd8ar0ywa4767y6xyak-example","","")],[],[],"x86_64-linux","/bin/sh",[],[("out","/gnu/store/kh7fais2zab22fd8ar0ywa4767y6xyak-example")])
(pk (read-derivation-from-file (derivation-file-name irssi-drv)))
;;; (#<derivation /gnu/store/zcgmhac8r4kdj2s6bcvcmhh4k35qvihx-irssi-1.4.3.drv => /gnu/store/v5pd69j3hjs1fck4b5p9hd91wc8yf5qx-irssi-1.4.3 7fb3798788c0>)
(call-with-input-file (derivation-file-name irssi-drv)
read-derivation)
;;; (#<derivation /gnu/store/zcgmhac8r4kdj2s6bcvcmhh4k35qvihx-irssi-1.4.3.drv => /gnu/store/v5pd69j3hjs1fck4b5p9hd91wc8yf5qx-irssi-1.4.3 7fb37ad19e10>)</code></pre><p>You can realise <code><derivation></code>s as store items using the <code>build-derivations</code>
procedure:</p><pre><code class="language-scheme">(use-modules (ice-9 ftw))
(define irssi-drv-out
(pk (derivation-output-path
(assoc-ref (derivation-outputs irssi-drv) "out"))))
;;; ("/gnu/store/v5pd69j3hjs1fck4b5p9hd91wc8yf5qx-irssi-1.4.3")
(pk (scandir irssi-drv-out))
;;; (#f)
(pk (with-store %store (build-derivations %store (list irssi-drv))))
;;; (#t)
(pk (scandir irssi-drv-out))
;;; (("." ".." "bin" "etc" "include" "lib" "share"))</code></pre><h1>Conclusion</h1><p>Derivations are one of Guix's most important concepts, but are fairly easy to
understand once you get past the obtuse <code>.drv</code> file format. They provide the
Guix daemon with the initial instructions that it uses to build store items
like packages, origins, and other file-likes such as <code>computed-file</code> and
<code>local-file</code>, which will be discussed in a future post!</p><p>To recap, a derivation contains the following fields:</p><ol><li><code>derivation-outputs</code>, describing the various output paths that the derivation
builds</li><li><code>derivation-inputs</code>, describing the other derivations that need to be built
before this one is</li><li><code>derivation-sources</code>, listing the non-derivation store items that the
derivation depends on</li><li><code>derivation-system</code>, specifying the system type a derivation will be compiled
for</li><li><code>derivation-builder</code>, the executable to run the build script with</li><li><code>derivation-builder-arguments</code>, arguments to pass to the builder</li><li><code>derivation-builder-environment-vars</code>, variables to set in the builder's
environment</li></ol><h4>About GNU Guix</h4><p><a href="https://guix.gnu.org">GNU Guix</a> is a transactional package manager and
an advanced distribution of the GNU system that <a href="https://www.gnu.org/distros/free-system-distribution-guidelines.html">respects user
freedom</a>.
Guix can be used on top of any system running the Hurd or the Linux
kernel, or it can be used as a standalone operating system distribution
for i686, x86_64, ARMv7, AArch64 and POWER9 machines.</p><p>In addition to standard package management features, Guix supports
transactional upgrades and roll-backs, unprivileged package management,
per-user profiles, and garbage collection. When used as a standalone
GNU/Linux distribution, Guix offers a declarative, stateless approach to
operating system configuration management. Guix is highly customizable
and hackable through <a href="https://www.gnu.org/software/guile">Guile</a>
programming interfaces and extensions to the
<a href="http://schemers.org">Scheme</a> language.</p>