B9 - A Benign VM-Build Tool

NOTE: The feature list is not completely implemented yet!

NOTE: Sometimes the CHANGELOG.md contains additional documentation!

Quick-Start

Mission Statement

B9 is an executable and a Haskell library, that consumes a Haskell term describing the generation of VM-Images.

A few core data types form an EDSL, and describe virtual machine images. B9 contains a library as well as an executable to interpret the EDSL.

Such a term can then be stored into a text file and is interpreted by a command line invokation.

Installation

Installation on NixOS

• As command line utility in current directory:

  $ nix-env -f https://github.com/sheyll/b9-vm-image-builder/archive/3.2.0.tar.gz -iA b9c
  

Runtime dependencies

To be able to use B9 install:

• Linux
• qemu
• ext4 tools
• genisoimage
• mtools
• vfat tools
• ssh
• rsync
• bash
• wget
• sudo (optional)

LibVirtLXC support

• lxc
• libvirt with lxc support (libvirt-driver-lxc, libvirt-daemon-lxc,...)
• virsh

B9 has been tested with libvirt version 1.2.12.

Make sure that all neccessary daemons, e.g. libvirtd.service, lxc.service,.. are active, that SELinux is configured correctly and that the nbd kernel module is loaded.

If neccessary create a libvirt network configuration, e.g. by using the GUI front-endvirt-manager.

Depending upon the libvirt and lxc configuration of the system it might be nessary to allow the user, that will execute b9c, password-less sudo for these commands:

• virsh
• rm
• cat
• cp
• mv

For SystemdNspawn support

• systemd

For Podman support (TODO)

• podman

For Docker support (TODO)

• docker

After installing B9 (either from a binary package or by building from source) all its glory is availbe through a single executable called b9c.

When b9c is started for the first time, it creates a configuration file in the users home directory called ~/.b9/b9.conf. The path to that file can be changed using command line arguments. Execute:

b9c -h

for a list of command line parameters and commands.

b9c command line arguments always follow this pattern:

b9c <global-options> <command> <command-options> -- <build-script-extra-args>

To enable B9 to work correctly on your machine edit the config file and make necessary adaptions.

General Overview

Use B9 to compile your software into a deployable set of Linux-VM- or configuration images, from a set of scripts and input files and templates.

The main goal of this tool is to provide a build tool to increase automation and reduce redundancy when creating configured, ready-to-run VM-images.

One big thing is that it can produce many machines and cloud-configs from a single build file, because build files can describe concrete as well as parameterized generators. It can create parameterized VM-Images by uploading (e.g. system-)files assembled by syntax aware template application and combination, all statically checked by during the build.

This sets B9 apart from e.g. cloud-init or other configuration management systems that provide configuration through user provided dynamic script-programs, which rely on the user to contain correct error handling.

The general idea is the same as in statically type programming languages: catch errors as early as possible without relying on the user to create a covering set of tests/error checks.

Certain sacrifies were made; there might be a steep laerning curve, but you will eventually get there. The tool at hand works stable and reliable.

All builds happen in isolation, i.e. by default in random build directories, and are cleaned on failure.

Also, B9 does not modify cached images. Work on VM-Images is always done on a copy of an image, and to speed things up, it is possible to explicitly use copy-on-write images.

B9 creates bootable virtual machine images, without necessarily using virtualization itself.

In essence B9 is a tool for creation, configuration and sharing of VM images and all peripheral artifacts, it creates:

• VMDK/QCOW2/Raw VM images
• Converted, extracted and resized copies of existing vm images
• Empty VM images with extended 4 file system
• Cloud-Config Images
• Text files from template files with variable interpolation
• Erlang OTP sys.config files

Make VM Disk Images

• Extract partitions from MRB Partitioned images
• Create, resuse, resize EXT-4 on Qcow2, vmdk or raw Images
• Run commands on images to create new image, similar to what docker build does,
  • Using libvirt-lxc
  • Using docker
  • Using systemd-nspawn

Manage Disk Images

• Cache images locally
• Distribute cached images via SCP

Assemble Cloud-init Configuration

• Merge YAML Expressions
• Merge Erlang Terms
• Load local files
• Load files from HTTP servers
• Support ${identifier} variable interpolation
• Create cloud-init
  • ISO images
  • Floppy images
  • yaml files

Input files

The input files can be in:

• Dhall format TODO
• Haskell values interpreted by the Read instances.

Usage as Library

• Use as a Haskell library

Configuration

B9 uses a .ini - style configuration file.

Incremental Builds

B9 uses shake so some degree of incremental build is available.

B9 build files

If you really need to write these file, you are basically f'ed.

For now, look at existing config files and read the sources, if anything, make sure to read at least the chapter Anger-Management before throwing stuff around.

More documentation is comming soon!

General Structure

A B9 configuration describes a single ArtifactGenerator. It generates files belonging to a VM, such as qcow2/raw/vmdk-image file(s) and e.g. cloud-init ISO image files.

Just to recap: a something.b9 build file is always ever only a mere ArtifactGenerator literal, no matter how many Let, Each, Artifacts, etc... you see flying around there.

Creating artifacts

To get any real artifact out of an artifact generator use the Artifact constructor. It takes 2 parameters an arbitrary id and a describtion of what the artifact consists of:

 Artifact (IID "some_instance_id")
          (VmImages ... | CloudInit ...)

An artifact can either be a (set of) VM-disk-image(s) likely in combination with some shell script to install software, etc or a static collection of files put on a cloud-init image(VFAT or ISO or directory).

Defining artifact generators that produce vm image files

To produce vm image files, e.g. with some software installed use the VmImages artifact generator. It has only 2 parameters:

 VmImages
    [ ... disk image targets ... ]
    ( ... installation script ...)

Of course it must be wrapped in an Artifact definition, so we get this structure:

 Artifact (IID "my_first_image")
   (VmImages [...] (...))

ImageTargets

The first argument to VmImages is a list of ImageTarget. Each describes a single VM-disk-image. The syntax is:

ImageTarget
  ImageDestination
  ImageSource
  MountPoint

• An ImageDestination specifies if/where to put the output image.
• An ImageSource specifies how the image is created or from where it is taken.
• A MountPoint specifies where to mount the image during the execution of an optional VmBuild-script.

Parameterized artifact generators

B9 supports $varnam variable interpolation in all strings anywhere in an ArtifactGenerator:

• All filenames and paths
• All id strings and names
• Template files included via e.g. Template
• In every string in VmScripts (e.g. in Run "${cmd}" ["${world}"])
• Also in all included template files (e.g. included via Template)

Parameters can be defined using Let, Each and special command line arguments.

To pass parameters via the command line, append them after the argument delimiter option -- which ends the list of regular b9c arguments:

b9c -v build -f file1.b9 .. -- arg_1 arg_2 ...

The parameters are bound to ${arg_1}, ${arg_2}, that is variables indicating the corresponding position on the command line.

To define variables using Let, write:

Let [key-value-pairs, ...]
    [artifactgenerators, ...]

All key-value bindings defined in the first list are available in every artifact generator contained in the second list (body).

A key-value binding, e.g. ("hostname", "www.acme.org"), consist of two strings on parens seperated by a , (comma). The left string is the key, and the right string is the value.

This ("webserver", "www.${domainname}") is an example to show that the value may also contain a variable reference. (Of course, only to variabled defined before)

Anger-Management

B9 build files contain a single literal ArtifactGenerator value in Haskell syntax. B9 currently 'parses' the config file without any error checking, so writing config files is VERY frustrating without some tricks:

Trick 1

Start with a working file and run

b9c reformat -f <filename>

after each modification. The reformat command only parses and - hence the name - (re-) formats/pretty-prints the files passed with -f options.

You will immediately know if a modification broke the file.

NOTE: If your build file refers to any ${arg_...} positional arguments pass them to reformat using -- followed by the argument list.

Trick 2

Obtain and build the sources of B9, start an interactive haskell shell with the B9 code loaded and try to paste the contents of the config file to see if ghci accepts it. Use the ghci macros :{ and :} to begin and end a multi-line input and paste the raw contents of the config file in question in between.

$ cabal install
$ cabal repl

... (many lines omitted) ...

*B9> :{
*B9| Artifact (IID "filer")
*B9|   (VmImages [ ImageTarget
*B9|                 (LocalFile (Image "EXPORT/machines/filer/disks/0.vmdk" Vmdk Ext4)
*B9|                             KeepSize)
*B9|                 (From "fedora-20-prod" KeepSize)
*B9|                 (MountPoint "/")
*B9|             , ImageTarget
*B9|                 (LocalFile (Image "EXPORT/machines/filer/disks/1.vmdk" Vmdk Ext4)
*B9|                             KeepSize)
*B9|                 (EmptyImage "audio_files" Ext4 Raw (ImageSize 64 GB))
*B9|                 (MountPoint "/export/lb/audio")
*B9|             ]
*B9|             (VmScript X86_64
*B9|               [ SharedDirectoryRO "./filer" (MountPoint "/mnt/build_root")
*B9|               , SharedDirectoryRO "../_common/upload" (MountPoint "/mnt/common")]
*B9|               (Begin
*B9|                  [ Run "dhclient" []
*B9|                  , In "/mnt/build_root" [ Run "./machine-" [] ]
*B9|                  , In "/mnt/common" [ Run "./post_export.sh" [] ]
*B9|                  ])))
*B9| :}

Artifact (IID "filer") (VmImages
[ImageTarget (LocalFile (Image "EXPORT/machines/filer/disks/0.vmdk" Vmdk Ext4) KeepSize)
(From "fedora-20-prod" KeepSize) (MountPoint "/"),ImageTarget
(LocalFile (Image "EXPORT/machines/filer/disks/1.vmdk" Vmdk Ext4) KeepSize)
(EmptyImage "audio_files" Ext4 Raw (ImageSize 64 GB)) (MountPoint "/export/lb/audio")]
(VmScript X86_64
[SharedDirectoryRO "./filer" (MountPoint "/mnt/build_root"),
SharedDirectoryRO "../_common/upload" (MountPoint "/mnt/common")]
(Begin [Run "dhclient" [],In "/mnt/build_root" [Run "./machine-" []],In
"/mnt/common" [Run "./post_export.sh" []]])))

B9 Config File Reference

This is an example of a B9 configuration file, by default found in ~/.b9/b9.conf and uses the ini-file format.

Some of the options can also be specified on the command line and in environment variables.

A path to an alternative config file can given on the command lines.

When the default config file does not exist, b9c will create it from default values.

This example is the current default configuration:

[global]
build_dir_root: /foo/binary
keep_temp_dirs: False
log_file: /tmp/b9.log
max_cached_shared_images: 2
repository: tilia
repository_cache: InB9UserDir "repo-cache"
unique_build_dirs: True
verbosity: LogInfo
default_timeout_seconds: 3600
timeout_factor: 4

[libvirt-lxc]
connection: lxc:///
emulator_path: /usr/lib/libvirt/libvirt_lxc
guest_capabilities: [CAP_MKNOD,CAP_SYS_ADMIN,CAP_SYS_CHROOT,CAP_SETGID,CAP_SETUID,CAP_NET_BIND_SERVICE,CAP_SETPCAP,CAP_SYS_PTRACE,CAP_SYS_MODULE]
guest_ram_size: RamSize 1 GB
network: Nothing
use_sudo: True

[podman]
guest_capabilities: [CAP_MKNOD,CAP_SYS_ADMIN,CAP_SYS_CHROOT,CAP_SETGID,CAP_SETUID,CAP_NET_BIND_SERVICE,CAP_SETPCAP,CAP_SYS_PTRACE,CAP_SYS_MODULE]
network: default

[systemdNspawn]
console: read-only
executable: /nix/store/bl92gd78ygdwmc94lrqy20ych0lrfis5-systemd-243.7/bin/systemd-nspawn
extra_args: "--user foo"
guest_capabilities: [CAP_MKNOD,CAP_SYS_ADMIN,CAP_SYS_CHROOT,CAP_SETGID,CAP_SETUID,CAP_NET_BIND_SERVICE,CAP_SETPCAP,CAP_SYS_PTRACE,CAP_SYS_MODULE]
max_lifetime_seconds: 14400
use_sudo: True

[docker]
guest_capabilities: [CAP_MKNOD,CAP_SYS_ADMIN,CAP_SYS_CHROOT,CAP_SETGID,CAP_SETUID,CAP_NET_BIND_SERVICE,CAP_SETPCAP,CAP_SYS_PTRACE,CAP_SYS_MODULE]
network: default 

The [global] Section

build_dir_root:

• Default: Nothing

TODO document this option.

keep_temp_dirs:

• Default: False

TODO document this option.

log_file:

• Default: Nothing

TODO document this option.

max_cached_shared_images:

• Default: Just 2

TODO document this option.

repository:

• Default: Nothing

TODO document this option.

repository_cache:

• Default: Just (InB9UserDir "repo-cache")

TODO document this option.

unique_build_dirs:

• Default: True

TODO document this option.

verbosity:

• Default: Just LogInfo

TODO document this option.

default_timeout_seconds

• optional default timeout, applied to all external calls:

timeout_factor

• optional factor(natural numbers) that every timeout value is multiplied by, for slower systems.

The [libvirt-lxc] Section

connection

• Default: lxc:///

TODO document this option.

emulator_path

• Default: Just "/usr/lib/libvirt/libvirt_lxc"

TODO document this option.

guest_capabilities

• Default: [CAP_MKNOD,CAP_SYS_ADMIN,CAP_SYS_CHROOT,CAP_SETGID,CAP_SETUID,CAP_NET_BIND_SERVICE,CAP_SETPCAP,CAP_SYS_PTRACE,CAP_SYS_MODULE]

TODO document this option.

guest_ram_size

• Default: RamSize 1 GB

TODO document this option.

network

• Default: Nothing

TODO document this option.

use_sudo

• Default: True

TODO document this option.

The [podman] Section

Since: 1.0.0

guest_capabilities

• Default: [CAP_MKNOD,CAP_SYS_ADMIN,CAP_SYS_CHROOT,CAP_SETGID,CAP_SETUID,CAP_NET_BIND_SERVICE,CAP_SETPCAP,CAP_SYS_PTRACE,CAP_SYS_MODULE]

TODO document this option.

network

• Default: Nothing

TODO document this option.

The [systemdNspawn] Section

Since: 1.0.0

console

• Default: read-only
• Valid Values:
  • read-only
  • interactive
  • passive
  • pipe

This corresponds to the --console=<...> parameter to systemd-nspawn.

If the value is set to interactive the execution of any VmScripts through systemd-nspawn will be interactive by using stdin, stdout and stderr of the b9c process.

executable

• Default: Nothing

TODO document this option.

extra_args

• Default: Nothing

TODO document this option.

guest_capabilities

• Default: [CAP_MKNOD,CAP_SYS_ADMIN,CAP_SYS_CHROOT,CAP_SETGID,CAP_SETUID,CAP_NET_BIND_SERVICE,CAP_SETPCAP,CAP_SYS_PTRACE,CAP_SYS_MODULE]

TODO document this option.

max_lifetime_seconds

• Default: Just 14400

TODO document this option.

use_sudo

• Default: True

TODO document this option.

The [docker] Section

Since: 1.0.0

guest_capabilities

• Default: [CAP_MKNOD,CAP_SYS_ADMIN,CAP_SYS_CHROOT,CAP_SETGID,CAP_SETUID,CAP_NET_BIND_SERVICE,CAP_SETPCAP,CAP_SYS_PTRACE,CAP_SYS_MODULE]

TODO document this option.

network

• Default: Nothing

TODO document this option.

ext4_attributes

• Default: []

List of options for mkfs.ext4 -O.

image_file_names_shortener_base_path

• Default: Nothing

If (Just "/path") is specified, images passed to libvirt are hard linked to a random subdirectory in /path/, such that the name will be short enough to match libvirtds file name length restriction. NOTE: The hard links have to be on the same file system as their targets.

Together with keep_temp_dirs the paths can be inspected, but also have to be manually removed.