Next: Invoking guix repl, Previous: The Store Monad, Up: Programming Interface [Contents][Index]
So we have “derivations”, which represent a sequence of build actions
to be performed to produce an item in the store (see Derivations).
These build actions are performed when asking the daemon to actually
build the derivations; they are run by the daemon in a container
(see Invoking guix-daemon).
It should come as no surprise that we like to write these build actions
in Scheme. When we do that, we end up with two strata of Scheme
code24: the “host code”—code that defines packages, talks
to the daemon, etc.—and the “build code”—code that actually
performs build actions, such as making directories, invoking
make, and so on (see Build Phases).
To describe a derivation and its build actions, one typically needs to
embed build code inside host code. It boils down to manipulating build
code as data, and the homoiconicity of Scheme—code has a direct
representation as data—comes in handy for that. But we need more than
the normal quasiquote mechanism in Scheme to construct build
expressions.
The (guix gexp) module implements G-expressions, a form of
S-expressions adapted to build expressions. G-expressions, or
gexps, consist essentially of three syntactic forms: gexp,
ungexp, and ungexp-splicing (or simply: #~,
#$, and #$@), which are comparable to
quasiquote, unquote, and unquote-splicing,
respectively (see quasiquote in GNU Guile Reference Manual). However, there are major differences:
This mechanism is not limited to package and derivation
objects: compilers able to “lower” other high-level objects to
derivations or files in the store can be defined,
such that these objects can also be inserted
into gexps. For example, a useful type of high-level objects that can be
inserted in a gexp is “file-like objects”, which make it easy to
add files to the store and to refer to them in
derivations and such (see local-file and plain-file
below).
To illustrate the idea, here is an example of a gexp:
(define build-exp
#~(begin
(mkdir #$output)
(chdir #$output)
(symlink (string-append #$coreutils "/bin/ls")
"list-files")))
This gexp can be passed to gexp->derivation; we obtain a
derivation that builds a directory containing exactly one symlink to
/gnu/store/…-coreutils-8.22/bin/ls:
(gexp->derivation "the-thing" build-exp)
As one would expect, the "/gnu/store/…-coreutils-8.22" string is
substituted to the reference to the coreutils package in the
actual build code, and coreutils is automatically made an input to
the derivation. Likewise, #$output (equivalent to (ungexp
output)) is replaced by a string containing the directory name of the
output of the derivation.
In a cross-compilation context, it is useful to distinguish between
references to the native build of a package—that can run on the
host—versus references to cross builds of a package. To that end, the
#+ plays the same role as #$, but is a reference to a
native package build:
(gexp->derivation "vi"
#~(begin
(mkdir #$output)
(mkdir (string-append #$output "/bin"))
(system* (string-append #+coreutils "/bin/ln")
"-s"
(string-append #$emacs "/bin/emacs")
(string-append #$output "/bin/vi")))
#:target "aarch64-linux-gnu")
In the example above, the native build of coreutils is used, so
that ln can actually run on the host; but then the
cross-compiled build of emacs is referenced.
Another gexp feature is imported modules: sometimes you want to be
able to use certain Guile modules from the “host environment” in the
gexp, so those modules should be imported in the “build environment”.
The with-imported-modules form allows you to express that:
(let ((build (with-imported-modules '((guix build utils))
#~(begin
(use-modules (guix build utils))
(mkdir-p (string-append #$output "/bin"))))))
(gexp->derivation "empty-dir"
#~(begin
#$build
(display "success!\n")
#t)))
In this example, the (guix build utils) module is automatically
pulled into the isolated build environment of our gexp, such that
(use-modules (guix build utils)) works as expected.
Usually you want the closure of the module to be imported—i.e.,
the module itself and all the modules it depends on—rather than just
the module; failing to do that, attempts to use the module will fail
because of missing dependent modules. The source-module-closure
procedure computes the closure of a module by looking at its source file
headers, which comes in handy in this case:
(use-modules (guix modules)) ;for 'source-module-closure' (with-imported-modules (source-module-closure '((guix build utils) (gnu build image))) (gexp->derivation "something-with-vms" #~(begin (use-modules (guix build utils) (gnu build image)) …)))
In the same vein, sometimes you want to import not just pure-Scheme
modules, but also “extensions” such as Guile bindings to C libraries
or other “full-blown” packages. Say you need the guile-json
package available on the build side, here’s how you would do it:
(use-modules (gnu packages guile)) ;for 'guile-json' (with-extensions (list guile-json) (gexp->derivation "something-with-json" #~(begin (use-modules (json)) …)))
The syntactic form to construct gexps is summarized below.
Return a G-expression containing exp. exp may contain one or more of the following forms:
#$obj(ungexp obj)Introduce a reference to obj. obj may have one of the
supported types, for example a package or a
derivation, in which case the ungexp form is replaced by its
output file name—e.g., "/gnu/store/…-coreutils-8.22.
If obj is a list, it is traversed and references to supported objects are substituted similarly.
If obj is another gexp, its contents are inserted and its dependencies are added to those of the containing gexp.
If obj is another kind of object, it is inserted as is.
#$obj:output(ungexp obj output)This is like the form above, but referring explicitly to the output of obj—this is useful when obj produces multiple outputs (see Packages with Multiple Outputs).
Sometimes a gexp unconditionally refers to the "out" output, but
the user of that gexp would still like to insert a reference to another
output. The gexp-input procedure aims to address that.
See gexp-input.
#+obj#+obj:output(ungexp-native obj)(ungexp-native obj output)Same as ungexp, but produces a reference to the native
build of obj when used in a cross compilation context.
#$output[:output](ungexp output [output])Insert a reference to derivation output output, or to the main output when output is omitted.
This only makes sense for gexps passed to gexp->derivation.
#$@lst(ungexp-splicing lst)Like the above, but splices the contents of lst inside the containing list.
#+@lst(ungexp-native-splicing lst)Like the above, but refers to native builds of the objects listed in lst.
G-expressions created by gexp or #~ are run-time objects
of the gexp? type (see below).
Mark the gexps defined in body… as requiring modules in their execution environment.
Each item in modules can be the name of a module, such as
(guix build utils), or it can be a module name, followed by an
arrow, followed by a file-like object:
`((guix build utils) (guix gcrypt) ((guix config) => ,(scheme-file "config.scm" #~(define-module …))))
In the example above, the first two modules are taken from the search path, and the last one is created from the given file-like object.
This form has lexical scope: it has an effect on the gexps directly defined in body…, but not on those defined, say, in procedures called from body….
Mark the gexps defined in body… as requiring
extensions in their build and execution environment.
extensions is typically a list of package objects such as those
defined in the (gnu packages guile) module.
Concretely, the packages listed in extensions are added to the load path while compiling imported modules in body…; they are also added to the load path of the gexp returned by body….
Return #t if obj is a G-expression.
G-expressions are meant to be written to disk, either as code building some derivation, or as plain files in the store. The monadic procedures below allow you to do that (see The Store Monad, for more information about monads).
%load-path] [#:effective-version "2.2"] [#:references-graphs #f] [#:allowed-references #f] [#:disallowed-references #f] [#:leaked-env-vars #f] [#:script-name (string-append name "-builder")] [#:deprecation-warnings #f] [#:local-build? #f] [#:substitutable? #t] [#:properties '()] [#:guile-for-build #f]Return a derivation name that runs exp (a gexp) with guile-for-build (a derivation) on system; exp is stored in a file called script-name. When target is true, it is used as the cross-compilation target triplet for packages referred to by exp.
modules is deprecated in favor of with-imported-modules.
Its meaning is to
make modules available in the evaluation context of exp;
modules is a list of names of Guile modules searched in
module-path to be copied in the store, compiled, and made available in
the load path during the execution of exp—e.g., ((guix
build utils) (guix build gnu-build-system)).
effective-version determines the string to use when adding extensions of
exp (see with-extensions) to the search path—e.g., "2.2".
graft? determines whether packages referred to by exp should be grafted when applicable.
When references-graphs is true, it must be a list of tuples of one of the following forms:
(file-name obj) (file-name obj output) (file-name gexp-input) (file-name store-item)
The right-hand-side of each element of references-graphs is automatically made an input of the build process of exp. In the build environment, each file-name contains the reference graph of the corresponding item, in a simple text format.
allowed-references must be either #f or a list of output names and packages.
In the latter case, the list denotes store items that the result is allowed to
refer to. Any reference to another store item will lead to a build error.
Similarly for disallowed-references, which can list items that must not be
referenced by the outputs.
deprecation-warnings determines whether to show deprecation warnings while
compiling modules. It can be #f, #t, or 'detailed.
The other arguments are as for derivation (see Derivations).
The local-file, plain-file, computed-file,
program-file, and scheme-file procedures below return
file-like objects. That is, when unquoted in a G-expression,
these objects lead to a file in the store. Consider this G-expression:
#~(system* #$(file-append glibc "/sbin/nscd") "-f" #$(local-file "/tmp/my-nscd.conf"))
The effect here is to “intern” /tmp/my-nscd.conf by copying it
to the store. Once expanded, for instance via
gexp->derivation, the G-expression refers to that copy under
/gnu/store; thus, modifying or removing the file in /tmp
does not have any effect on what the G-expression does.
plain-file can be used similarly; it differs in that the file
content is directly passed as a string.
Return an object representing local file file to add to the store; this object can be used in a gexp. If file is a literal string denoting a relative file name, it is looked up relative to the source file where it appears; if file is not a literal string, it is looked up relative to the current working directory at run time. file will be added to the store under name–by default the base name of file.
When recursive? is true, the contents of file are added recursively; if file designates a flat file and recursive? is true, its contents are added, and its permission bits are kept.
When recursive? is true, call (select? file
stat) for each directory entry, where file is the entry’s
absolute file name and stat is the result of lstat; exclude
entries for which select? does not return true.
file can be wrapped in the assume-valid-file-name syntactic
keyword. When this is done, there will not be a warning when
local-file is used with a non-literal path. The path is still
looked up relative to the current working directory at run time.
Wrapping is done like this:
(define alice-key-file-path "alice.pub") ;; ... (local-file (assume-valid-file-name alice-key-file-path))
file can be wrapped in the assume-source-relative-file-name
syntactic keyword. When this is done, the file name will be looked up
relative to the source file where it appears even when it is not a
string literal.
This is the declarative counterpart of the interned-file monadic
procedure (see interned-file).
Return an object representing a text file called name with the given content (a string or a bytevector) to be added to the store.
This is the declarative counterpart of text-file.
Return an object representing the store item name, a file or
directory computed by gexp. When local-build? is true (the
default), the derivation is built locally. options is a list of
additional arguments to pass to gexp->derivation.
This is the declarative counterpart of gexp->derivation.
Return an executable script name that runs exp using guile, with exp’s imported modules in its search path. Look up exp’s modules in module-path.
The example below builds a script that simply invokes the ls
command:
(use-modules (guix gexp) (gnu packages base)) (gexp->script "list-files" #~(execl #$(file-append coreutils "/bin/ls") "ls"))
When “running” it through the store (see run-with-store), we obtain a derivation that produces an
executable file /gnu/store/…-list-files along these lines:
#!/gnu/store/…-guile-2.0.11/bin/guile -ds !# (execl "/gnu/store/…-coreutils-8.22"/bin/ls" "ls")
Return an object representing the executable store item name that runs gexp. guile is the Guile package used to execute that script. Imported modules of gexp are looked up in module-path.
This is the declarative counterpart of gexp->script.
Return a derivation that builds a file name containing exp. When splice? is true, exp is considered to be a list of expressions that will be spliced in the resulting file.
When set-load-path? is true, emit code in the resulting file to
set %load-path and %load-compiled-path to honor
exp’s imported modules. Look up exp’s modules in
module-path.
The resulting file holds references to all the dependencies of exp or a subset thereof.
Return an object representing the Scheme file name that contains exp. guile is the Guile package used to produce that file.
This is the declarative counterpart of gexp->file.
Return as a monadic value a derivation that builds a text file containing all of text. text may list, in addition to strings, objects of any type that can be used in a gexp: packages, derivations, local file objects, etc. The resulting store file holds references to all these.
This variant should be preferred over text-file anytime the file
to create will reference items from the store. This is typically the
case when building a configuration file that embeds store file names,
like this:
(define (profile.sh)
;; Return the name of a shell script in the store that
;; initializes the 'PATH' environment variable.
(text-file* "profile.sh"
"export PATH=" coreutils "/bin:"
grep "/bin:" sed "/bin\n"))
In this example, the resulting /gnu/store/…-profile.sh file will reference coreutils, grep, and sed, thereby preventing them from being garbage-collected during its lifetime.
Return an object representing store file name containing text. text is a sequence of strings and file-like objects, as in:
(mixed-text-file "profile"
"export PATH=" coreutils "/bin:" grep "/bin")
This is the declarative counterpart of text-file*.
Return a <computed-file> that builds a directory containing all of files.
Each item in files must be a two-element list where the first element is the
file name to use in the new directory, and the second element is a gexp
denoting the target file. Here’s an example:
(file-union "etc"
`(("hosts" ,(plain-file "hosts"
"127.0.0.1 localhost"))
("bashrc" ,(plain-file "bashrc"
"alias ls='ls --color=auto'"))))
This yields an etc directory containing these two files.
Return a directory that is the union of things, where things is a list of file-like objects denoting directories. For example:
(directory-union "guile+emacs" (list guile emacs))
yields a directory that is the union of the guile and emacs packages.
Return a file-like object that expands to the concatenation of obj and suffix, where obj is a lowerable object and each suffix is a string.
As an example, consider this gexp:
(gexp->script "run-uname"
#~(system* #$(file-append coreutils
"/bin/uname")))
The same effect could be achieved with:
(gexp->script "run-uname"
#~(system* (string-append #$coreutils
"/bin/uname")))
There is one difference though: in the file-append case, the
resulting script contains the absolute file name as a string, whereas in
the second case, the resulting script contains a (string-append
…) expression to construct the file name at run time.
Bind system to the currently targeted system—e.g.,
"x86_64-linux"—within body.
In the second case, additionally bind target to the current
cross-compilation target—a GNU triplet such as
"arm-linux-gnueabihf"—or #f if we are not
cross-compiling.
let-system is useful in the occasional case where the object
spliced into the gexp depends on the target system, as in this example:
#~(system* #+(let-system system (cond ((string-prefix? "armhf-" system) (file-append qemu "/bin/qemu-system-arm")) ((string-prefix? "x86_64-" system) (file-append qemu "/bin/qemu-system-x86_64")) (else (error "dunno!")))) "-net" "user" #$image)
This macro is similar to the parameterize form for
dynamically-bound parameters (see Parameters in GNU
Guile Reference Manual). The key difference is that it takes effect
when the file-like object returned by exp is lowered to a
derivation or store item.
A typical use of with-parameters is to force the system in effect
for a given object:
(with-parameters ((%current-system "i686-linux"))
coreutils)
The example above returns an object that corresponds to the i686 build
of Coreutils, regardless of the current value of %current-system.
Return a gexp input record for the given output of file-like
object obj, with #:native? determining whether this is a
native reference (as with ungexp-native) or not.
This procedure is helpful when you want to pass a reference to a specific output of an object to some procedure that may not know about that output. For example, assume you have this procedure, which takes one file-like object:
(define (make-symlink target)
(computed-file "the-symlink"
#~(symlink #$target #$output)))
Here make-symlink can only ever refer to the default output of
target—the "out" output (see Packages with Multiple Outputs). To have it refer to, say, the "lib" output of the
hwloc package, you can call it like so:
(make-symlink (gexp-input hwloc "lib"))
You can also compose it like any other file-like object:
(make-symlink
(file-append (gexp-input hwloc "lib") "/lib/libhwloc.so"))
Of course, in addition to gexps embedded in “host” code, there are
also modules containing build tools. To make it clear that they are
meant to be used in the build stratum, these modules are kept in the
(guix build …) name space.
Internally, high-level objects are lowered, using their compiler,
to either derivations or store items. For instance, lowering a package
yields a derivation, and lowering a plain-file yields a store
item. This is achieved using the lower-object monadic procedure.
Return as a value in %store-monad the derivation or store item
corresponding to obj for system, cross-compiling for
target if target is true. obj must be an object that
has an associated gexp compiler, such as a <package>.
Sometimes, it may be useful to convert a G-exp into a S-exp. For
example, some linters (see Invoking guix lint) peek into the build
phases of a package to detect potential problems. This conversion can
be achieved with this procedure. However, some information can be lost
in the process. More specifically, lowerable objects will be silently
replaced with some arbitrary object – currently the list
(*approximate*), but this may change.
The term stratum in this context was coined by Manuel Serrano et al. in the context of their work on Hop. Oleg Kiselyov, who has written insightful essays and code on this topic, refers to this kind of code generation as staging.
Next: Invoking guix repl, Previous: The Store Monad, Up: Programming Interface [Contents][Index]