Next: , Previous: , Up: Programming Interface   [Contents][Index]


9.7 Build Utilities

As soon as you start writing non-trivial package definitions (see Defining Packages) or other build actions (see G-Expressions), you will likely start looking for helpers for “shell-like” actions—creating directories, copying and deleting files recursively, manipulating build phases, and so on. The (guix build utils) module provides such utility procedures.

Most build systems load (guix build utils) (see Build Systems). Thus, when writing custom build phases for your package definitions, you can usually assume those procedures are in scope.

When writing G-expressions, you can import (guix build utils) on the “build side” using with-imported-modules and then put it in scope with the use-modules form (see Using Guile Modules in GNU Guile Reference Manual):

(with-imported-modules '((guix build utils))  ;import it
  (computed-file "empty-tree"
                 #~(begin
                     ;; Put it in scope.
                     (use-modules (guix build utils))

                     ;; Happily use its 'mkdir-p' procedure.
                     (mkdir-p (string-append #$output "/a/b/c")))))

The remainder of this section is the reference for most of the utility procedures provided by (guix build utils).

9.7.1 Dealing with Store File Names

This section documents procedures that deal with store file names.

Scheme Procedure: %store-directory

Return the directory name of the store.

Scheme Procedure: store-file-name? file

Return true if file is in the store.

Scheme Procedure: strip-store-file-name file

Strip the /gnu/store and hash from file, a store file name. The result is typically a "package-version" string.

Scheme Procedure: package-name->name+version name

Given name, a package name like "foo-0.9.1b", return two values: "foo" and "0.9.1b". When the version part is unavailable, name and #f are returned. The first hyphen followed by a digit is considered to introduce the version part.

9.7.2 File Types

The procedures below deal with files and file types.

Scheme Procedure: directory-exists? dir

Return #t if dir exists and is a directory.

Scheme Procedure: executable-file? file

Return #t if file exists and is executable.

Return #t if file is a symbolic link (aka. a “symlink”).

Scheme Procedure: elf-file? file
Scheme Procedure: ar-file? file
Scheme Procedure: gzip-file? file

Return #t if file is, respectively, an ELF file, an ar archive (such as a .a static library), or a gzip file.

Scheme Procedure: reset-gzip-timestamp file [#:keep-mtime? #t]

If file is a gzip file, reset its embedded timestamp (as with gzip --no-name) and return true. Otherwise return #f. When keep-mtime? is true, preserve file’s modification time.

9.7.3 File Manipulation

The following procedures and macros help create, modify, and delete files. They provide functionality comparable to common shell utilities such as mkdir -p, cp -r, rm -r, and sed. They complement Guile’s extensive, but low-level, file system interface (see POSIX in GNU Guile Reference Manual).

Scheme Syntax: with-directory-excursion directory body

Run body with directory as the process’s current directory.

Essentially, this macro changes the current directory to directory before evaluating body, using chdir (see Processes in GNU Guile Reference Manual). It changes back to the initial directory when the dynamic extent of body is left, be it via normal procedure return or via a non-local exit such as an exception.

Scheme Procedure: mkdir-p dir

Create directory dir and all its ancestors.

Scheme Procedure: install-file file directory

Create directory if it does not exist and copy file in there under the same name.

Scheme Procedure: make-file-writable file

Make file writable for its owner.

Scheme Procedure: copy-recursively source destination [#:log (current-output-port)] [#:follow-symlinks? #f] [#:copy-file copy-file] [#:keep-mtime? #f] [#:keep-permissions? #t]

Copy source directory to destination. Follow symlinks if follow-symlinks? is true; otherwise, just preserve them. Call copy-file to copy regular files. When keep-mtime? is true, keep the modification time of the files in source on those of destination. When keep-permissions? is true, preserve file permissions. Write verbose output to the log port.

Scheme Procedure: delete-file-recursively dir [#:follow-mounts? #f]

Delete dir recursively, like rm -rf, without following symlinks. Don’t follow mount points either, unless follow-mounts? is true. Report but ignore errors.

Scheme Syntax: substitute* file ((regexp match-var…) body…) …

Substitute regexp in file by the string returned by body. body is evaluated with each match-var bound to the corresponding positional regexp sub-expression. For example:

(substitute* file
  (("hello")
   "good morning\n")
  (("foo([a-z]+)bar(.*)$" all letters end)
   (string-append "baz" letters end)))

Here, anytime a line of file contains hello, it is replaced by good morning. Anytime a line of file matches the second regexp, all is bound to the complete match, letters is bound to the first sub-expression, and end is bound to the last one.

When one of the match-var is _, no variable is bound to the corresponding match substring.

Alternatively, file may be a list of file names, in which case they are all subject to the substitutions.

Be careful about using $ to match the end of a line; by itself it won’t match the terminating newline of a line.

9.7.4 File Search

This section documents procedures to search and filter files.

Scheme Procedure: file-name-predicate regexp

Return a predicate that returns true when passed a file name whose base name matches regexp.

Scheme Procedure: find-files dir [pred] [#:stat lstat] [#:directories? #f] [#:fail-on-error? #f]

Return the lexicographically sorted list of files under dir for which pred returns true. pred is passed two arguments: the absolute file name, and its stat buffer; the default predicate always returns true. pred can also be a regular expression, in which case it is equivalent to (file-name-predicate pred). stat is used to obtain file information; using lstat means that symlinks are not followed. If directories? is true, then directories will also be included. If fail-on-error? is true, raise an exception upon error.

Here are a few examples where we assume that the current directory is the root of the Guix source tree:

;; List all the regular files in the current directory.
(find-files ".")
 ("./.dir-locals.el" "./.gitignore" )

;; List all the .scm files under gnu/services.
(find-files "gnu/services" "\\.scm$")
 ("gnu/services/admin.scm" "gnu/services/audio.scm" )

;; List ar files in the current directory.
(find-files "." (lambda (file stat) (ar-file? file)))
 ("./libformat.a" "./libstore.a" )
Scheme Procedure: which program

Return the complete file name for program as found in $PATH, or #f if program could not be found.

Scheme Procedure: search-input-file inputs name
Scheme Procedure: search-input-directory inputs name

Return the complete file name for name as found in inputs; search-input-file searches for a regular file and search-input-directory searches for a directory. If name could not be found, an exception is raised.

Here, inputs must be an association list like inputs and native-inputs as available to build phases (see Build Phases).

Here is a (simplified) example of how search-input-file is used in a build phase of the wireguard-tools package:

(add-after 'install 'wrap-wg-quick
  (lambda* (#:key inputs outputs #:allow-other-keys)
    (let ((coreutils (string-append (assoc-ref inputs "coreutils")
                                    "/bin")))
      (wrap-program (search-input-file outputs "bin/wg-quick")
        #:sh (search-input-file inputs "bin/bash")
        `("PATH" ":" prefix ,(list coreutils))))))

9.7.5 Program Invocation

You’ll find handy procedures to spawn processes in this module, essentially convenient wrappers around Guile’s system* (see system* in GNU Guile Reference Manual).

Scheme Procedure: invoke program args

Invoke program with the given args. Raise an &invoke-error exception if the exit code is non-zero; otherwise return #t.

The advantage compared to system* is that you do not need to check the return value. This reduces boilerplate in shell-script-like snippets for instance in package build phases.

Scheme Procedure: invoke-error? c

Return true if c is an &invoke-error condition.

Scheme Procedure: invoke-error-program c
Scheme Procedure: invoke-error-arguments c
Scheme Procedure: invoke-error-exit-status c
Scheme Procedure: invoke-error-term-signal c
Scheme Procedure: invoke-error-stop-signal c

Access specific fields of c, an &invoke-error condition.

Scheme Procedure: report-invoke-error c [port]

Report to port (by default the current error port) about c, an &invoke-error condition, in a human-friendly way.

Typical usage would look like this:

(use-modules (srfi srfi-34) ;for 'guard'
             (guix build utils))

(guard (c ((invoke-error? c)
           (report-invoke-error c)))
  (invoke "date" "--imaginary-option"))

-| command "date" "--imaginary-option" failed with status 1
Scheme Procedure: invoke/quiet program args

Invoke program with args and capture program’s standard output and standard error. If program succeeds, print nothing and return the unspecified value; otherwise, raise a &message error condition that includes the status code and the output of program.

Here’s an example:

(use-modules (srfi srfi-34) ;for 'guard'
             (srfi srfi-35) ;for 'message-condition?'
             (guix build utils))

(guard (c ((message-condition? c)
           (display (condition-message c))))
  (invoke/quiet "date")  ;all is fine
  (invoke/quiet "date" "--imaginary-option"))

-| 'date --imaginary-option' exited with status 1; output follows:

    date: unrecognized option '--imaginary-option'
    Try 'date --help' for more information.

9.7.6 Build Phases

The (guix build utils) also contains tools to manipulate build phases as used by build systems (see Build Systems). Build phases are represented as association lists or “alists” (see Association Lists in GNU Guile Reference Manual) where each key is a symbol naming the phase and the associated value is a procedure (see Build Phases).

Guile core and the (srfi srfi-1) module both provide tools to manipulate alists. The (guix build utils) module complements those with tools written with build phases in mind.

Scheme Syntax: modify-phases phases clause

Modify phases sequentially as per each clause, which may have one of the following forms:

(delete old-phase-name)
(replace old-phase-name new-phase)
(add-before old-phase-name new-phase-name new-phase)
(add-after old-phase-name new-phase-name new-phase)

Where every phase-name above is an expression evaluating to a symbol, and new-phase an expression evaluating to a procedure.

The example below is taken from the definition of the grep package. It adds a phase to run after the install phase, called fix-egrep-and-fgrep. That phase is a procedure (lambda* is for anonymous procedures) that takes a #:outputs keyword argument and ignores extra keyword arguments (see Optional Arguments in GNU Guile Reference Manual, for more on lambda* and optional and keyword arguments.) The phase uses substitute* to modify the installed egrep and fgrep scripts so that they refer to grep by its absolute file name:

(modify-phases %standard-phases
  (add-after 'install 'fix-egrep-and-fgrep
    ;; Patch 'egrep' and 'fgrep' to execute 'grep' via its
    ;; absolute file name instead of searching for it in $PATH.
    (lambda* (#:key outputs #:allow-other-keys)
      (let* ((out (assoc-ref outputs "out"))
             (bin (string-append out "/bin")))
        (substitute* (list (string-append bin "/egrep")
                           (string-append bin "/fgrep"))
          (("^exec grep")
           (string-append "exec " bin "/grep")))))))

In the example below, phases are modified in two ways: the standard configure phase is deleted, presumably because the package does not have a configure script or anything similar, and the default install phase is replaced by one that manually copies the executable files to be installed:

(modify-phases %standard-phases
  (delete 'configure)      ;no 'configure' script
  (replace 'install
    (lambda* (#:key outputs #:allow-other-keys)
      ;; The package's Makefile doesn't provide an "install"
      ;; rule so do it by ourselves.
      (let ((bin (string-append (assoc-ref outputs "out")
                                "/bin")))
        (install-file "footswitch" bin)
        (install-file "scythe" bin)))))

9.7.7 Wrappers

It is not unusual for a command to require certain environment variables to be set for proper functioning, typically search paths (see Search Paths). Failing to do that, the command might fail to find files or other commands it relies on, or it might pick the “wrong” ones—depending on the environment in which it runs. Examples include:

For a package writer, the goal is to make sure commands always work the same rather than depend on some external settings. One way to achieve that is to wrap commands in a thin script that sets those environment variables, thereby ensuring that those run-time dependencies are always found. The wrapper would be used to set PATH, GUILE_LOAD_PATH, or QT_PLUGIN_PATH in the examples above.

To ease that task, the (guix build utils) module provides a couple of helpers to wrap commands.

Scheme Procedure: wrap-program program [#:sh sh] [#:rest variables]

Make a wrapper for program. variables should look like this:

'(variable delimiter position list-of-directories)

where delimiter is optional. : will be used if delimiter is not given.

For example, this call:

(wrap-program "foo"
              '("PATH" ":" = ("/gnu/.../bar/bin"))
              '("CERT_PATH" suffix ("/gnu/.../baz/certs"
                                    "/qux/certs")))

will copy foo to .foo-real and create the file foo with the following contents:

#!location/of/bin/bash
export PATH="/gnu/.../bar/bin"
export CERT_PATH="$CERT_PATH${CERT_PATH:+:}/gnu/.../baz/certs:/qux/certs"
exec -a $0 location/of/.foo-real "$@"

If program has previously been wrapped by wrap-program, the wrapper is extended with definitions for variables. If it is not, sh will be used as the interpreter.

Scheme Procedure: wrap-script program [#:guile guile] [#:rest variables]

Wrap the script program such that variables are set first. The format of variables is the same as in the wrap-program procedure. This procedure differs from wrap-program in that it does not create a separate shell script. Instead, program is modified directly by prepending a Guile script, which is interpreted as a comment in the script’s language.

Special encoding comments as supported by Python are recreated on the second line.

Note that this procedure can only be used once per file as Guile scripts are not supported.


Next: , Previous: , Up: Programming Interface   [Contents][Index]