Next: , Previous: , Up: 设置后台进程   [Contents][Index]


2.4.2 使用任务下发设施

When desired, the build daemon can offload derivation builds to other machines running Guix, using the offload build hook7. When that feature is enabled, a list of user-specified build machines is read from /etc/guix/machines.scm; every time a build is requested, for instance via guix build, the daemon attempts to offload it to one of the machines that satisfy the constraints of the derivation, in particular its system types—e.g., x86_64-linux. A single machine can have multiple system types, either because its architecture natively supports it, via emulation (see Transparent Emulation with QEMU), or both. Missing prerequisites for the build are copied over SSH to the target machine, which then proceeds with the build; upon success the output(s) of the build are copied back to the initial machine. The offload facility comes with a basic scheduler that attempts to select the best machine. The best machine is chosen among the available machines based on criteria such as:

  1. The availability of a build slot. A build machine can have as many build slots (connections) as the value of the parallel-builds field of its build-machine object.
  2. Its relative speed, as defined via the speed field of its build-machine object.
  3. Its load. The normalized machine load must be lower than a threshold value, configurable via the overload-threshold field of its build-machine object.
  4. Disk space availability. More than a 100 MiB must be available.

/etc/guix/machines.scm文件通常是这样的:

(list (build-machine
        (name "eightysix.example.org")
        (systems (list "x86_64-linux" "i686-linux"))
        (host-key "ssh-ed25519 AAAAC3Nza…")
        (user "bob")
        (speed 2.))     ;incredibly fast!

      (build-machine
        (name "armeight.example.org")
        (systems (list "aarch64-linux"))
        (host-key "ssh-rsa AAAAB3Nza…")
        (user "alice")
        (private-key
         (string-append (getenv "HOME")
                        "/.ssh/identity-for-guix"))))

In the example above we specify a list of two build machines, one for the x86_64 and i686 architectures and one for the aarch64 architecture.

事实上,这个文件–并不意外地–是一个Scheme文件,当下发钩子被启动时执行。它的返回值必须是一个包含build-machine对象的列表。虽然这个例子展示的是一个固定的列表,你可以想象,使用DNS-SD来返回一个包含从局域网内发现的构建机器的列表,see Guile-Avahi in 在Guile Scheme程序里使用Avahibuild-machine数据类型的详细信息如下。

数据类型: build-machine

这个数据类型表示后台进程可以下发构建任务的构建机器。重要的项有:

名字

远程机器的主机名。

systems

The system types the remote machine supports—e.g., (list "x86_64-linux" "i686-linux").

用户

通过SSH连接远程机器时使用的用户帐号。注意,SSH密钥能被密码保护,以支持无交互的登录。

主机公钥

这必须是机器的OpenSSH格式的SSH公钥。这是用来在连接机器时认证身份的。它是一个像这样的长字符串:

ssh-ed25519 AAAAC3NzaC…mde+UhL hint@example.org

如果这个机器正在运行OpenSSH后台进程,sshd,那么主机公钥可以在/etc/ssh/ssh_host_ed25519_key.pub找到。

如果这个机器正在运行GNU lsh,lshd,那么主机公钥可以在/etc/lsh/host-key.pub或类似的位置找到。它可以通过lsh-export-key命令转换成OpenSSH格式(see Converting keys in LSH用户手册):

$ lsh-export-key --openssh < /etc/lsh/host-key.pub
ssh-rsa AAAAB3NzaC1yc2EAAAAEOp8FoQAAAQEAs1eB46LV…

一些可选的项:

port(默认值:22

机器上的SSH服务器的端口号。

private-key(默认值:~root/.ssh/id_rsa

连接机器时使用的SSH私钥,OpenSSH格式。这个私钥不能被密码保护。

注意,默认值是root帐号的私钥。使用默认值时请确保它存在。

compression(默认值:"zlib@openssh.com,zlib"
compression-level(默认值:3

SSH压缩算法和压缩级别。

下发任务依赖SSH压缩来减少传输文件到构建机器时使用的带宽。

daemon-socket(默认值:"/var/guix/daemon-socket/socket"

那台机器上的guix-daemon监听的Unix套接字文件名。

overload-threshold (default: 0.6)

The load threshold above which a potential offload machine is disregarded by the offload scheduler. The value roughly translates to the total processor usage of the build machine, ranging from 0.0 (0%) to 1.0 (100%). It can also be disabled by setting overload-threshold to #f.

parallel-builds(默认值:1

那台机器上可以并行运行的构建任务数量。

speed(默认值:1.0

一个相对的速度值。下发调度器会偏好速度更快的机器。

features'()

一个表示机器支持的功能的字符串列表。例如,"kvm"表示机器有KVM Linux模块和相关的硬件支持。Derivation可以通过名字请求需要的功能,然后被分发到匹配的机器的任务队列里。

guix命令必须在构建机器的搜素路径里。你可以通过这个命令检查:

ssh build-machine guix repl --version

machines.scm到位后,还有一件要做的事。如上所述,下发任务时会在机器的仓库之间传输文件。为此,你需要在每台机器上生成一个密钥对,以使后台进程可以从仓库导出签名后的文件包(see 调用guix archive):

# guix archive --generate-key

每台构建机器都必须认证主机器的公钥,从而接收从主机器接收的仓库文件:

# guix archive --authorize < master-public-key.txt

类似的,主机器必须认证每台构建机器的公钥:

所有这些有关公钥的繁琐事宜都是为了表达主服务器和构建服务器之间成对的互相信任关系。具体地,当主机器从构建机器接收文件时(反之亦然),它的构建后台进程可以确保文件是原样的,没有被篡改,并且被认证的公钥签名过。

为了测试你的设置是否能正常工作,在主节点上运行这个命令:

# guix offload test

This will attempt to connect to each of the build machines specified in /etc/guix/machines.scm, make sure Guix is available on each machine, attempt to export to the machine and import from it, and report any error in the process.

如果你希望用别的文件测试,只需要在命令行指定它:

# guix offload test machines-qualif.scm

最后,你可以像这样只测试机器列表里名字匹配某个正则表达式的子集:

# guix offload test machines.scm '\.gnu\.org$'

若想展示所有构建主机的当前负载,在主节点上运行这个命令:

# guix offload status

Footnotes

(7)

This feature is available only when Guile-SSH is present.


Next: , Previous: , Up: 设置后台进程   [Contents][Index]