8.12 G-Expressions

So we have “derivations”, which represent a sequence of build actions to be performed to produce an item in the store (see Деривации). These build actions are performed when asking the daemon to actually build the derivations; they are run by the daemon in a container (see Вызов guix-daemon).

It should come as no surprise that we like to write these build actions in Scheme. When we do that, we end up with two strata of Scheme code²⁴: the “host code”—code that defines packages, talks to the daemon, etc.—and the “build code”—code that actually performs build actions, such as making directories, invoking make, and so on (see Фазы сборки).

To describe a derivation and its build actions, one typically needs to embed build code inside host code. It boils down to manipulating build code as data, and the homoiconicity of Scheme—code has a direct representation as data—comes in handy for that. But we need more than the normal quasiquote mechanism in Scheme to construct build expressions.

The (guix gexp) module implements G-expressions, a form of S-expressions adapted to build expressions. G-expressions, or gexps, consist essentially of three syntactic forms: gexp, ungexp, and ungexp-splicing (or simply: #~, #$, and #$@), which are comparable to quasiquote, unquote, and unquote-splicing, respectively (see quasiquote in GNU Guile Reference Manual). However, there are major differences:

• Gexps are meant to be written to a file and run or manipulated by other processes.
• When a high-level object such as a package or derivation is unquoted inside a gexp, the result is as if its output file name had been introduced.
• Gexps carry information about the packages or derivations they refer to, and these dependencies are automatically added as inputs to the build processes that use them.

This mechanism is not limited to package and derivation objects: compilers able to “lower” other high-level objects to derivations or files in the store can be defined, such that these objects can also be inserted into gexps. For example, a useful type of high-level objects that can be inserted in a gexp is “file-like objects”, which make it easy to add files to the store and to refer to them in derivations and such (see local-file and plain-file below).

To illustrate the idea, here is an example of a gexp:

(define build-exp
  #~(begin
      (mkdir #$output)
      (chdir #$output)
      (symlink (string-append #$coreutils "/bin/ls")
               "list-files")))

This gexp can be passed to gexp->derivation; we obtain a derivation that builds a directory containing exactly one symlink to /gnu/store/…-coreutils-8.22/bin/ls:

(gexp->derivation "the-thing" build-exp)

As one would expect, the "/gnu/store/…-coreutils-8.22" string is substituted to the reference to the coreutils package in the actual build code, and coreutils is automatically made an input to the derivation. Likewise, #$output (equivalent to (ungexp output)) is replaced by a string containing the directory name of the output of the derivation.

In a cross-compilation context, it is useful to distinguish between references to the native build of a package—that can run on the host—versus references to cross builds of a package. To that end, the #+ plays the same role as #$, but is a reference to a native package build:

(gexp->derivation "vi"
   #~(begin
       (mkdir #$output)
       (mkdir (string-append #$output "/bin"))
       (system* (string-append #+coreutils "/bin/ln")
                "-s"
                (string-append #$emacs "/bin/emacs")
                (string-append #$output "/bin/vi")))
   #:target "aarch64-linux-gnu")

In the example above, the native build of coreutils is used, so that ln can actually run on the host; but then the cross-compiled build of emacs is referenced.

Another gexp feature is imported modules: sometimes you want to be able to use certain Guile modules from the “host environment” in the gexp, so those modules should be imported in the “build environment”. The with-imported-modules form allows you to express that:

(let ((build (with-imported-modules '((guix build utils))
               #~(begin
                   (use-modules (guix build utils))
                   (mkdir-p (string-append #$output "/bin"))))))
  (gexp->derivation "empty-dir"
                    #~(begin
                        #$build
                        (display "success!\n")
                        #t)))

In this example, the (guix build utils) module is automatically pulled into the isolated build environment of our gexp, such that (use-modules (guix build utils)) works as expected.

Usually you want the closure of the module to be imported—i.e., the module itself and all the modules it depends on—rather than just the module; failing to do that, attempts to use the module will fail because of missing dependent modules. The source-module-closure procedure computes the closure of a module by looking at its source file headers, which comes in handy in this case:

(use-modules (guix modules))   ;for 'source-module-closure'

(with-imported-modules (source-module-closure
                         '((guix build utils)
                           (gnu build image)))
  (gexp->derivation "something-with-vms"
                    #~(begin
                        (use-modules (guix build utils)
                                     (gnu build image))
                        …)))

In the same vein, sometimes you want to import not just pure-Scheme modules, but also “extensions” such as Guile bindings to C libraries or other “full-blown” packages. Say you need the guile-json package available on the build side, here’s how you would do it:

(use-modules (gnu packages guile))  ;for 'guile-json'

(with-extensions (list guile-json)
  (gexp->derivation "something-with-json"
                    #~(begin
                        (use-modules (json))
                        …)))

The syntactic form to construct gexps is summarized below.

Macro: #~exp ¶

Macro: (gexp exp)

Return a G-expression containing exp. exp may contain one or more of the following forms:

#$obj

(ungexp obj)

Introduce a reference to obj. obj may have one of the supported types, for example a package or a derivation, in which case the ungexp form is replaced by its output file name—e.g., "/gnu/store/…-coreutils-8.22.

If obj is a list, it is traversed and references to supported objects are substituted similarly.

If obj is another gexp, its contents are inserted and its dependencies are added to those of the containing gexp.

If obj is another kind of object, it is inserted as is.

#$obj:output

(ungexp obj output)

This is like the form above, but referring explicitly to the output of obj—this is useful when obj produces multiple outputs (see Пакеты со множественным выходом).

Sometimes a gexp unconditionally refers to the "out" output, but the user of that gexp would still like to insert a reference to another output. The gexp-input procedure aims to address that. See gexp-input.

#+obj

#+obj:output

(ungexp-native obj)

(ungexp-native obj output)

Same as ungexp, but produces a reference to the native build of obj when used in a cross compilation context.

#$output[:output]

(ungexp output [output])

Insert a reference to derivation output output, or to the main output when output is omitted.

This only makes sense for gexps passed to gexp->derivation.

#$@lst

(ungexp-splicing lst)

Like the above, but splices the contents of lst inside the containing list.

#+@lst

(ungexp-native-splicing lst)

Like the above, but refers to native builds of the objects listed in lst.

G-expressions created by gexp or #~ are run-time objects of the gexp? type (see below).

Macro: with-imported-modules modules body…

Mark the gexps defined in body… as requiring modules in their execution environment.

Each item in modules can be the name of a module, such as (guix build utils), or it can be a module name, followed by an arrow, followed by a file-like object:

`((guix build utils)
  (guix gcrypt)
  ((guix config) => ,(scheme-file "config.scm"
                                  #~(define-module …))))

In the example above, the first two modules are taken from the search path, and the last one is created from the given file-like object.

This form has lexical scope: it has an effect on the gexps directly defined in body…, but not on those defined, say, in procedures called from body….

Macro: with-extensions extensions body…

Mark the gexps defined in body… as requiring extensions in their build and execution environment. extensions is typically a list of package objects such as those defined in the (gnu packages guile) module.

Concretely, the packages listed in extensions are added to the load path while compiling imported modules in body…; they are also added to the load path of the gexp returned by body….

Procedure: gexp? obj

Return #t if obj is a G-expression.

G-expressions are meant to be written to disk, either as code building some derivation, or as plain files in the store. The monadic procedures below allow you to do that (see Устройство склада, for more information about monads).

Monadic Procedure: gexp->derivation name exp [#:system (%current-system)] [#:target #f] [#:graft? #t]  [#:hash #f]

[#:hash-algo #f]  [#:recursive? #f] [#:env-vars ’()] [#:modules ’()]  [#:module-path %load-path]  [#:effective-version "2.2"]  [#:references-graphs #f] [#:allowed-references #f]  [#:disallowed-references #f]  [#:leaked-env-vars #f]  [#:script-name (string-append name "-builder")]  [#:deprecation-warnings #f]  [#:local-build? #f] [#:substitutable? #t]  [#:properties ’()] [#:guile-for-build #f] Return a derivation name that runs exp (a gexp) with guile-for-build (a derivation) on system; exp is stored in a file called script-name. When target is true, it is used as the cross-compilation target triplet for packages referred to by exp.

modules is deprecated in favor of with-imported-modules. Its meaning is to make modules available in the evaluation context of exp; modules is a list of names of Guile modules searched in module-path to be copied in the store, compiled, and made available in the load path during the execution of exp—e.g., ((guix build utils) (guix build gnu-build-system)).

effective-version determines the string to use when adding extensions of exp (see with-extensions) to the search path—e.g., "2.2".

graft? determines whether packages referred to by exp should be grafted when applicable.

When references-graphs is true, it must be a list of tuples of one of the following forms:

(file-name obj)
(file-name obj output)
(file-name gexp-input)
(file-name store-item)

The right-hand-side of each element of references-graphs is automatically made an input of the build process of exp. In the build environment, each file-name contains the reference graph of the corresponding item, in a simple text format.

allowed-references must be either #f or a list of output names and packages. In the latter case, the list denotes store items that the result is allowed to refer to. Any reference to another store item will lead to a build error. Similarly for disallowed-references, which can list items that must not be referenced by the outputs.

deprecation-warnings determines whether to show deprecation warnings while compiling modules. It can be #f, #t, or 'detailed.

The other arguments are as for derivation (see Деривации).

The local-file, plain-file, computed-file, program-file, and scheme-file procedures below return file-like objects. That is, when unquoted in a G-expression, these objects lead to a file in the store. Consider this G-expression:

#~(system* #$(file-append glibc "/sbin/nscd") "-f"
           #$(local-file "/tmp/my-nscd.conf"))

The effect here is to “intern” /tmp/my-nscd.conf by copying it to the store. Once expanded, for instance via gexp->derivation, the G-expression refers to that copy under /gnu/store; thus, modifying or removing the file in /tmp does not have any effect on what the G-expression does. plain-file can be used similarly; it differs in that the file content is directly passed as a string.

Procedure: local-file file [name] [#:recursive? #f] [#:select? (const #t)]

Return an object representing local file file to add to the store; this object can be used in a gexp. If file is a literal string denoting a relative file name, it is looked up relative to the source file where it appears; if file is not a literal string, it is looked up relative to the current working directory at run time. file will be added to the store under name–by default the base name of file.

When recursive? is true, the contents of file are added recursively; if file designates a flat file and recursive? is true, its contents are added, and its permission bits are kept.

When recursive? is true, call (select? file stat) for each directory entry, where file is the entry’s absolute file name and stat is the result of lstat; exclude entries for which select? does not return true.

file can be wrapped in the assume-valid-file-name syntactic keyword. When this is done, there will not be a warning when local-file is used with a non-literal path. The path is still looked up relative to the current working directory at run time. Wrapping is done like this:

(define alice-key-file-path "alice.pub")
;; ...
(local-file (assume-valid-file-name alice-key-file-path))

file can be wrapped in the assume-source-relative-file-name syntactic keyword. When this is done, the file name will be looked up relative to the source file where it appears even when it is not a string literal.

This is the declarative counterpart of the interned-file monadic procedure (see interned-file).

Procedure: plain-file name content

Return an object representing a text file called name with the given content (a string or a bytevector) to be added to the store.

This is the declarative counterpart of text-file.

Procedure: computed-file name gexp [#:local-build? #t] [#:options '()]

Return an object representing the store item name, a file or directory computed by gexp. When local-build? is true (the default), the derivation is built locally. options is a list of additional arguments to pass to gexp->derivation.

This is the declarative counterpart of gexp->derivation.

Monadic Procedure: gexp->script name exp [#:guile (default-guile)] [#:module-path %load-path]  [#:system

(%current-system)] [#:target #f] Return an executable script name that runs exp using guile, with exp’s imported modules in its search path. Look up exp’s modules in module-path.

The example below builds a script that simply invokes the ls command:

(use-modules (guix gexp) (gnu packages base))

(gexp->script "list-files"
              #~(execl #$(file-append coreutils "/bin/ls")
                       "ls"))

When “running” it through the store (see run-with-store), we obtain a derivation that produces an executable file /gnu/store/…-list-files along these lines:

#!/gnu/store/…-guile-2.0.11/bin/guile -ds
!#
(execl "/gnu/store/…-coreutils-8.22"/bin/ls" "ls")

Procedure: program-file name exp [#:guile #f] [#:module-path %load-path]

Return an object representing the executable store item name that runs gexp. guile is the Guile package used to execute that script. Imported modules of gexp are looked up in module-path.

This is the declarative counterpart of gexp->script.

Monadic Procedure: gexp->file name exp [#:set-load-path? #t] [#:module-path %load-path]  [#:splice? #f]  [#:guile

(default-guile)] Return a derivation that builds a file name containing exp. When splice? is true, exp is considered to be a list of expressions that will be spliced in the resulting file.

When set-load-path? is true, emit code in the resulting file to set %load-path and %load-compiled-path to honor exp’s imported modules. Look up exp’s modules in module-path.

The resulting file holds references to all the dependencies of exp or a subset thereof.

Procedure: scheme-file name exp [#:splice? #f] [#:guile #f] [#:set-load-path? #t] Return an object representing the Scheme

file name that contains exp. guile is the Guile package used to produce that file.

This is the declarative counterpart of gexp->file.

Monadic Procedure: text-file* name text …

Return as a monadic value a derivation that builds a text file containing all of text. text may list, in addition to strings, objects of any type that can be used in a gexp: packages, derivations, local file objects, etc. The resulting store file holds references to all these.

This variant should be preferred over text-file anytime the file to create will reference items from the store. This is typically the case when building a configuration file that embeds store file names, like this:

(define (profile.sh)
  ;; Return the name of a shell script in the store that
  ;; initializes the 'PATH' environment variable.
  (text-file* "profile.sh"
              "export PATH=" coreutils "/bin:"
              grep "/bin:" sed "/bin\n"))

In this example, the resulting /gnu/store/…-profile.sh file will reference coreutils, grep, and sed, thereby preventing them from being garbage-collected during its lifetime.

Procedure: mixed-text-file name text …

Return an object representing store file name containing text. text is a sequence of strings and file-like objects, as in:

(mixed-text-file "profile"
                 "export PATH=" coreutils "/bin:" grep "/bin")

Это декларативный аналог text-file*.

Procedure: file-union name files

Return a <computed-file> that builds a directory containing all of files. Each item in files must be a two-element list where the first element is the file name to use in the new directory, and the second element is a gexp denoting the target file. Here’s an example:

(file-union "etc"
            `(("hosts" ,(plain-file "hosts"
                                    "127.0.0.1 localhost"))
              (\"bashrc\" ,(plain-file "bashrc"
                                     "alias ls='ls --color=auto'"))))

This yields an etc directory containing these two files.

Procedure: directory-union name things

Return a directory that is the union of things, where things is a list of file-like objects denoting directories. For example:

(directory-union "guile+emacs" (list guile emacs))

yields a directory that is the union of the guile and emacs packages.

Procedure: file-append obj suffix …

Return a file-like object that expands to the concatenation of obj and suffix, where obj is a lowerable object and each suffix is a string.

В качестве примера рассмотрим этот gexp:

(gexp->script "run-uname"
              #~(system* #$(file-append coreutils
                                        "/bin/uname")))

Такого же эффекта можно добиться с помощью:

(gexp->script "run-uname"
              #~(system* (string-append #$coreutils
                                        "/bin/uname")))

There is one difference though: in the file-append case, the resulting script contains the absolute file name as a string, whereas in the second case, the resulting script contains a (string-append …) expression to construct the file name at run time.

Macro: let-system system body…

Macro: let-system (system target) body…

Привязать system к текущей целевой системе—например, \"x86_64-linux\"—в body.

In the second case, additionally bind target to the current cross-compilation target—a GNU triplet such as "arm-linux-gnueabihf"—or #f if we are not cross-compiling.

let-system is useful in the occasional case where the object spliced into the gexp depends on the target system, as in this example:

#~(system*
   #+(let-system system
       (cond ((string-prefix? "armhf-" system)
              (file-append qemu "/bin/qemu-system-arm"))
             ((string-prefix? "x86_64-" system)
              (file-append qemu "/bin/qemu-system-x86_64"))
             (else
              (error "dunno!"))))
   "-net" "user" #$image)

Macro: with-parameters ((parameter value) …) exp

This macro is similar to the parameterize form for dynamically-bound parameters (see Parameters in GNU Guile Reference Manual). The key difference is that it takes effect when the file-like object returned by exp is lowered to a derivation or store item.

A typical use of with-parameters is to force the system in effect for a given object:

(with-parameters ((%current-system "i686-linux"))
  coreutils)

The example above returns an object that corresponds to the i686 build of Coreutils, regardless of the current value of %current-system.

Procedure: gexp-input obj [output] [#:native? #f]

Return a gexp input record for the given output of file-like object obj, with #:native? determining whether this is a native reference (as with ungexp-native) or not.

This procedure is helpful when you want to pass a reference to a specific output of an object to some procedure that may not know about that output. For example, assume you have this procedure, which takes one file-like object:

(define (make-symlink target)
  (computed-file "the-symlink"
                 #~(symlink #$target #$output)))

Here make-symlink can only ever refer to the default output of target—the "out" output (see Пакеты со множественным выходом). To have it refer to, say, the "lib" output of the hwloc package, you can call it like so:

(make-symlink (gexp-input hwloc "lib"))

You can also compose it like any other file-like object:

(make-symlink
  (file-append (gexp-input hwloc "lib") "/lib/libhwloc.so"))

Of course, in addition to gexps embedded in “host” code, there are also modules containing build tools. To make it clear that they are meant to be used in the build stratum, these modules are kept in the (guix build …) name space.

Internally, high-level objects are lowered, using their compiler, to either derivations or store items. For instance, lowering a package yields a derivation, and lowering a plain-file yields a store item. This is achieved using the lower-object monadic procedure.

Monadic Procedure: lower-object obj [system] [#:target #f] Return as a value in %store-monad the derivation or

store item corresponding to obj for system, cross-compiling for target if target is true. obj must be an object that has an associated gexp compiler, such as a <package>.

Procedure: gexp->approximate-sexp gexp

Sometimes, it may be useful to convert a G-exp into a S-exp. For example, some linters (see Вызов guix lint) peek into the build phases of a package to detect potential problems. This conversion can be achieved with this procedure. However, some information can be lost in the process. More specifically, lowerable objects will be silently replaced with some arbitrary object – currently the list (*approximate*), but this may change.