Next: , Previous: , Up: Programming Interface   [Contents][Index]


8.7 Build Utilities

As soon as you start writing non-trivial package definitions (see Defining Packages) or other build actions (see G-Expressions), you will likely start looking for helpers for “shell-like” actions—creating directories, copying and deleting files recursively, manipulating build phases, and so on. The (guix build utils) module provides such utility procedures.

Most build systems load (guix build utils) (see Build Systems). Thus, when writing custom build phases for your package definitions, you can usually assume those procedures are in scope.

When writing G-expressions, you can import (guix build utils) on the “build side” using with-imported-modules and then put it in scope with the use-modules form (see Using Guile Modules in GNU Guile Reference Manual):

(with-imported-modules '((guix build utils))  ;import it
  (computed-file "empty-tree"
                 #~(begin
                     ;; Put it in scope.
                     (use-modules (guix build utils))

                     ;; Happily use its 'mkdir-p' procedure.
                     (mkdir-p (string-append #$output "/a/b/c")))))

The remainder of this section is the reference for most of the utility procedures provided by (guix build utils).

8.7.1 Dealing with Store File Names

This section documents procedures that deal with store file names.

Procedure: %store-directory

Return the directory name of the store.

Procedure: store-file-name? file

Return true if file is in the store.

Procedure: strip-store-file-name file

Strip the /gnu/store and hash from file, a store file name. The result is typically a "package-version" string.

Procedure: package-name->name+version name

Given name, a package name like "foo-0.9.1b", return two values: "foo" and "0.9.1b". When the version part is unavailable, name and #f are returned. The first hyphen followed by a digit is considered to introduce the version part.

8.7.2 File Types

The procedures below deal with files and file types.

Procedure: directory-exists? dir

Return #t if dir exists and is a directory.

Procedure: executable-file? file

Return #t if file exists and is executable.

Return #t if file is a symbolic link (aka. a “symlink”).

Procedure: elf-file? file
Procedure: ar-file? file
Procedure: gzip-file? file

Return #t if file is, respectively, an ELF file, an ar archive (such as a .a static library), or a gzip file.

Procedure: reset-gzip-timestamp file [#:keep-mtime? #t]

If file is a gzip file, reset its embedded timestamp (as with gzip --no-name) and return true. Otherwise return #f. When keep-mtime? is true, preserve file’s modification time.

8.7.3 File Manipulation

The following procedures and macros help create, modify, and delete files. They provide functionality comparable to common shell utilities such as mkdir -p, cp -r, rm -r, and sed. They complement Guile’s extensive, but low-level, file system interface (see POSIX in GNU Guile Reference Manual).

Macro: with-directory-excursion directory body …

Run body with directory as the process’s current directory.

Essentially, this macro changes the current directory to directory before evaluating body, using chdir (see Processes in GNU Guile Reference Manual). It changes back to the initial directory when the dynamic extent of body is left, be it via normal procedure return or via a non-local exit such as an exception.

Procedure: mkdir-p dir

Create directory dir and all its ancestors.

Procedure: install-file file directory

Create directory if it does not exist and copy file in there under the same name.

Procedure: make-file-writable file

Make file writable for its owner.

Procedure: copy-recursively source destination [#:log (current-output-port)] [#:follow-symlinks? #f] [#:copy-file copy-file] [#:keep-mtime? #f] [#:keep-permissions? #t]

[#:select? (const #t)] Copy source directory to destination. Follow symlinks if follow-symlinks? is true; otherwise, just preserve them. Call copy-file to copy regular files. Call select?, taking two arguments, file and stat, for each entry in source, where file is the entry’s absolute file name and stat is the result of lstat (or stat if follow-symlinks? is true); exclude entries for which select? does not return true. When keep-mtime? is true, keep the modification time of the files in source on those of destination. When keep-permissions? is true, preserve file permissions. Write verbose output to the log port.

Procedure: delete-file-recursively dir [#:follow-mounts? #f]

Delete dir recursively, like rm -rf, without following symlinks. Don’t follow mount points either, unless follow-mounts? is true. Report but ignore errors.

Macro: substitute* file ((regexp match-var…) body…) …

Substitute regexp in file by the string returned by body. body is evaluated with each match-var bound to the corresponding positional regexp sub-expression. For example:

(substitute* file
  (("hello")
   "good morning\n")
  (("foo([a-z]+)bar(.*)$" all letters end)
   (string-append "baz" letters end)))

Here, anytime a line of file contains hello, it is replaced by good morning. Anytime a line of file matches the second regexp, all is bound to the complete match, letters is bound to the first sub-expression, and end is bound to the last one.

When one of the match-var is _, no variable is bound to the corresponding match substring.

Alternatively, file may be a list of file names, in which case they are all subject to the substitutions.

Be careful about using $ to match the end of a line; by itself it won’t match the terminating newline of a line. For example, to match a whole line ending with a backslash, one needs a regex like "(.*)\\\\\n$".

8.7.5 Program Invocation

You’ll find handy procedures to spawn processes in this module, essentially convenient wrappers around Guile’s system* (see system* in GNU Guile Reference Manual).

Procedure: invoke program args…

Invoke program with the given args. Raise an &invoke-error exception if the exit code is non-zero; otherwise return #t.

The advantage compared to system* is that you do not need to check the return value. This reduces boilerplate in shell-script-like snippets for instance in package build phases.

Procedure: invoke-error? c

Return true if c is an &invoke-error condition.

Procedure: invoke-error-program c
Procedure: invoke-error-arguments c
Procedure: invoke-error-exit-status c
Procedure: invoke-error-term-signal c
Procedure: invoke-error-stop-signal c

Access specific fields of c, an &invoke-error condition.

Procedure: report-invoke-error c [port]

Report to port (by default the current error port) about c, an &invoke-error condition, in a human-friendly way.

Typical usage would look like this:

(use-modules (srfi srfi-34) ;for 'guard'
             (guix build utils))

(guard (c ((invoke-error? c)
           (report-invoke-error c)))
  (invoke "date" "--imaginary-option"))

-| command "date" "--imaginary-option" failed with status 1
Procedure: invoke/quiet program args…

Invoke program with args and capture program’s standard output and standard error. If program succeeds, print nothing and return the unspecified value; otherwise, raise a &message error condition that includes the status code and the output of program.

Here’s an example:

(use-modules (srfi srfi-34) ;for 'guard'
             (srfi srfi-35) ;for 'message-condition?'
             (guix build utils))

(guard (c ((message-condition? c)
           (display (condition-message c))))
  (invoke/quiet "date")  ;all is fine
  (invoke/quiet "date" "--imaginary-option"))

-| 'date --imaginary-option' exited with status 1; output follows:

    date: unrecognized option '--imaginary-option'
    Try 'date --help' for more information.

8.7.6 Build Phases

The (guix build utils) also contains tools to manipulate build phases as used by build systems (see Build Systems). Build phases are represented as association lists or “alists” (see Association Lists in GNU Guile Reference Manual) where each key is a symbol naming the phase and the associated value is a procedure (see Build Phases).

Guile core and the (srfi srfi-1) module both provide tools to manipulate alists. The (guix build utils) module complements those with tools written with build phases in mind.

Macro: modify-phases phases clause…

Modify phases sequentially as per each clause, which may have one of the following forms:

(delete old-phase-name)
(replace old-phase-name new-phase)
(add-before old-phase-name new-phase-name new-phase)
(add-after old-phase-name new-phase-name new-phase)

Where every phase-name above is an expression evaluating to a symbol, and new-phase an expression evaluating to a procedure.

The example below is taken from the definition of the grep package. It adds a phase to run after the install phase, called fix-egrep-and-fgrep. That phase is a procedure (lambda* is for anonymous procedures) that takes a #:outputs keyword argument and ignores extra keyword arguments (see Optional Arguments in GNU Guile Reference Manual, for more on lambda* and optional and keyword arguments.) The phase uses substitute* to modify the installed egrep and fgrep scripts so that they refer to grep by its absolute file name:

(modify-phases %standard-phases
  (add-after 'install 'fix-egrep-and-fgrep
    ;; Patch 'egrep' and 'fgrep' to execute 'grep' via its
    ;; absolute file name instead of searching for it in $PATH.
    (lambda* (#:key outputs #:allow-other-keys)
      (let* ((out (assoc-ref outputs "out"))
             (bin (string-append out "/bin")))
        (substitute* (list (string-append bin "/egrep")
                           (string-append bin "/fgrep"))
          (("^exec grep")
           (string-append "exec " bin "/grep")))))))

In the example below, phases are modified in two ways: the standard configure phase is deleted, presumably because the package does not have a configure script or anything similar, and the default install phase is replaced by one that manually copies the executable files to be installed:

(modify-phases %standard-phases
  (delete 'configure)      ;no 'configure' script
  (replace 'install
    (lambda* (#:key outputs #:allow-other-keys)
      ;; The package's Makefile doesn't provide an "install"
      ;; rule so do it by ourselves.
      (let ((bin (string-append (assoc-ref outputs "out")
                                "/bin")))
        (install-file "footswitch" bin)
        (install-file "scythe" bin)))))

8.7.7 Wrappers

It is not unusual for a command to require certain environment variables to be set for proper functioning, typically search paths (see Search Paths). Failing to do that, the command might fail to find files or other commands it relies on, or it might pick the “wrong” ones—depending on the environment in which it runs. Examples include:

For a package writer, the goal is to make sure commands always work the same rather than depend on some external settings. One way to achieve that is to wrap commands in a thin script that sets those environment variables, thereby ensuring that those run-time dependencies are always found. The wrapper would be used to set PATH, GUILE_LOAD_PATH, or QT_PLUGIN_PATH in the examples above.

To ease that task, the (guix build utils) module provides a couple of helpers to wrap commands.

Procedure: wrap-program program [#:sh sh] [#:rest variables]

Make a wrapper for program. variables should look like this:

'(variable delimiter position list-of-directories)

where delimiter is optional. : will be used if delimiter is not given.

For example, this call:

(wrap-program "foo"
              '("PATH" ":" = ("/gnu/.../bar/bin"))
              '("CERT_PATH" suffix ("/gnu/.../baz/certs"
                                    "/qux/certs")))

will copy foo to .foo-real and create the file foo with the following contents:

#!location/of/bin/bash
export PATH="/gnu/.../bar/bin"
export CERT_PATH="$CERT_PATH${CERT_PATH:+:}/gnu/.../baz/certs:/qux/certs"
exec -a $0 location/of/.foo-real "$@"

If program has previously been wrapped by wrap-program, the wrapper is extended with definitions for variables. If it is not, sh will be used as the interpreter.

Procedure: wrap-script program [#:guile guile] [#:rest variables]

Wrap the script program such that variables are set first. The format of variables is the same as in the wrap-program procedure. This procedure differs from wrap-program in that it does not create a separate shell script. Instead, program is modified directly by prepending a Guile script, which is interpreted as a comment in the script’s language.

Special encoding comments as supported by Python are recreated on the second line.

Note that this procedure can only be used once per file as Guile scripts are not supported.


Next: Search Paths, Previous: Build Phases, Up: Programming Interface   [Contents][Index]