| Authors: | Brian C. Lane <bcl@redhat.com> |
|---|
lorax-composer is an API server that allows you to build disk images usingBlueprints to describe the package versions to be installed into the image.It is compatible with the Weldr project’s bdcs-api REST protocol. Moreinformation on Weldr can be found on the Weldr blog.
Behind the scenes it uses livemedia-creator andAnaconda to handle theinstallation and configuration of the images.
Important Things To Note¶
- SELinux must be in Permissive mode. Anaconda requires SELinux be in permissive modefor image creation to work correctly. You can either edit the setting in the/etc/sysconfig/selinux file, or run setenforce 0 before starting lorax-composer.
- Some output types require packages from the RHEL 7 Optional repository. See theRed Hat Enterprise Linux 7 documentationfor information on how to enable it. Otherwise you will see image creation fail todepsolve even if the blueprint itself is correct.
- All image types lock the root account, except for live-iso. You will need to eitheruse one of the Customizations methods for setting a ssh key/password, install apackage that creates a user, or use something like cloud-init to setup access atboot time.
Installation¶
The best way to install lorax-composer is to use sudo dnf installlorax-composer composer-cli, this will setup the weldr user and install thesystemd socket activation service. You will then need to enable it with sudosystemctl enable lorax-composer.socket && sudo systemctl startlorax-composer.socket. This will leave the server off until the first requestis made. Systemd will then launch the server and it will remain running untilthe system is rebooted. This will cause some delay in responding to the firstrequest from the UI or composer-cli.
Note
If you want lorax-composer to respond immediately to the first request you canstart and enable lorax-composer.service instead of lorax-composer.socket
Quickstart¶
- Create a weldr user and group by running useradd weldr
- Remove any pre-existing socket directory with rm -rf /run/weldr/A new directory with correct permissions will be created the first time the server runs.
- Enable the socket activation with systemctl enable lorax-composer.socket&& sudo systemctl start lorax-composer.socket.
NOTE: You can also run it directly with lorax-composer /path/to/blueprints. However,lorax-composer does not react well to being started both on the command line and viasocket activation at the same time. It is therefore recommended that you run it directlyon the command line only for testing or development purposes. For real use or developmentof other projects that simply use the API, you should stick to socket activation only.
The /path/to/blueprints/ directory is where the blueprints’ git repo willbe created, and all the blueprints created with the /api/v0/blueprints/newroute will be stored. If there are blueprint .toml files in the top levelof the directory they will be imported into the blueprint git storage whenlorax-composer starts.
Logs¶
Logs are stored under /var/log/lorax-composer/ and include all consolemessages as well as extra debugging info and API requests.
Security¶
Some security related issues that you should be aware of before running lorax-composer:
- One of the API server threads needs to retain root privileges in order to run Anaconda.
- SELinux must be set to Permissive or disabled to allow livemedia-creator to run Anaconda.
- Only allow authorized users access to the weldr group and socket.
Since Anaconda kickstarts are used there is the possibility that a user couldinject commands into a blueprint that would result in the kickstart executingarbitrary code on the host. Only authorized users should be allowed to buildimages using lorax-composer.
How it Works¶
The server runs as root, and as weldr. Communication with it is via a unixdomain socket (/run/weldr/api.socket by default). The directory and socketare owned by root:weldr so that any user in the weldr group can use the APIto control lorax-composer.
At startup the server will check for the correct permissions andownership of a pre-existing directory, or it will create a new one if itdoesn’t exist. The socket path and group owner’s name can be changed from thecmdline by passing it the --socket and --group arguments.
It will then drop root privileges for the API thread and run as the weldruser. The queue and compose thread still runs as root because it needs to beable to mount/umount files and run Anaconda.
Composing Images¶
The welder-web GUI project can be used to constructblueprints and create composes using a web browser.
Or use the command line with composer-cli.
Blueprints¶
Blueprints are simple text files in TOML format that describewhich packages, and what versions, to install into the image. They can also define a limited setof customizations to make to the final image.
Example blueprints can be found in the lorax-composer test suite, with a simple onelooking like this:
name = "base"description = "A base system with bash"version = "0.0.1"[[packages]]name = "bash"version = "4.4.*"
The name field is the name of the blueprint. It can contain spaces, but they will be converted to -when it is written to disk. It should be short and descriptive.
description can be a longer description of the blueprint, it is only used for display purposes.
version is a semver compatible version number. Ifa new blueprint is uploaded with the same version the server willautomatically bump the PATCH level of the version. If the versiondoesn’t match it will be used as is. eg. Uploading a blueprint with versionset to 0.1.0 when the existing blueprint version is 0.0.1 willresult in the new blueprint being stored as version 0.1.0.
[[packages]] and [[modules]]¶
These entries describe the package names and matching version glob to be installed into the image.
The names must match the names exactly, and the versions can be an exact matchor a filesystem-like glob of the version using * wildcards and ?character matching.
NOTE: As of lorax-composer-29.2-1 the versions are not used for depsolving,that is planned for a future release. And currently there are no differencesbetween packages and modules in lorax-composer.
[[groups]]¶
These entries describe a group of packages to be installed into the image. Package groups aredefined in the repository metadata. Each group has a descriptive name used primarily for displayin user interfaces and an ID more commonly used in kickstart files. Here, the ID is the expectedway of listing a group.
Groups have three different ways of categorizing their packages: mandatory, default, and optional.For purposes of blueprints, mandatory and default packages will be installed. There is no mechanismfor selecting optional packages.
Customizations¶
The [customizations] section can be used to configure the hostname of the final image. eg.:
[customizations]hostname = "baseimage"
This is optional and may be left out to use the defaults.
[customizations.kernel]¶
This allows you to append arguments to the bootloader’s kernel commandline. This will not have anyeffect on tar or ext4-filesystem images since they do not include a bootloader.
For example:
[customizations.kernel]append = "nosmt=force"
[[customizations.sshkey]]¶
Set an existing user’s ssh key in the final image:
[[customizations.sshkey]]user = "root"key = "PUBLIC SSH KEY"
The key will be added to the user’s authorized_keys file.
[[customizations.user]]¶
Add a user to the image, and/or set their ssh key.All fields for this section are optional except for the name, here is a complete example:
[[customizations.user]]name = "admin"description = "Administrator account"password = "$6$CHO2$3rN8eviE2t50lmVyBYihTgVRHcaecmeCk31L..."key = "PUBLIC SSH KEY"home = "/srv/widget/"shell = "/usr/bin/bash"groups = ["widget", "users", "wheel"]uid = 1200gid = 1200
If the password starts with $6$, $5$, or $2b$ it will be stored asan encrypted password. Otherwise it will be treated as a plain text password.
[[customizations.group]]¶
Add a group to the image. name is required and gid is optional:
[[customizations.group]]name = "widget"gid = 1130
[customizations.timezone]¶
Customizing the timezone and the NTP servers to use for the system:
[customizations.timezone]timezone = "US/Eastern"ntpservers = ["0.north-america.pool.ntp.org", "1.north-america.pool.ntp.org"]
The values supported by timezone can be listed by running timedatectl list-timezones.
If no timezone is setup the system will default to using UTC. The ntp servers are alsooptional and will default to using the distribution defaults which are fine for most uses.
In some image types there are already NTP servers setup, eg. Google cloud image, and theycannot be overridden because they are required to boot in the selected environment. But thetimezone will be updated to the one selected in the blueprint.
[customizations.locale]¶
Customize the locale settings for the system:
[customizations.locale]languages = ["en_US.UTF-8"]keyboard = "us"
The values supported by languages can be listed by running localectl list-locales fromthe command line.
The values supported by keyboard can be listed by running localectl list-keymaps fromthe command line.
Multiple languages can be added. The first one becomes theprimary, and the others are added as secondary. One or the other of languagesor keyboard must be included (or both) in the section.
[customizations.firewall]¶
By default the firewall blocks all access except for services that enable their ports explicitly,like sshd. This command can be used to open other ports or services. Ports are configured usingthe port:protocol format:
[customizations.firewall]ports = ["22:tcp", "80:tcp", "imap:tcp", "53:tcp", "53:udp"]
Numeric ports, or their names from /etc/services can be used in the ports enabled/disabled lists.
The blueprint settings extend any existing settings in the image templates, so if sshd isalready enabled it will extend the list of ports with the ones listed by the blueprint.
If the distribution uses firewalld you can specify services listed by firewall-cmd --get-servicesin a customizations.firewall.services section:
[customizations.firewall.services]enabled = ["ftp", "ntp", "dhcp"]disabled = ["telnet"]
Remember that the firewall.services are different from the names in /etc/services.
Both are optional, if they are not used leave them out or set them to an empty list []. If youonly want the default firewall setup this section can be omitted from the blueprint.
NOTE: The Google and OpenStack templates explicitly disable the firewall for their environment.This cannot be overridden by the blueprint.
[customizations.services]¶
This section can be used to control which services are enabled at boot time.Some image types already have services enabled or disabled in order for theimage to work correctly, and cannot be overridden. eg. ami requiressshd, chronyd, and cloud-init. Without them the image will notboot. Blueprint services are added to, not replacing, the list already in thetemplates, if any.
[customizations.services]enabled = [“sshd”, “cockpit”, “httpd.service”]disabled = [“postfix”, “telnetd”]
Note
The service names are systemd service units. You can only specify the unitname, eg. httpd, or the full service name, eg. httpd.service – butnot other systemd unit files. Note that this is different from newerreleases where you can specify any systemd unit file.
Adding Output Types¶
livemedia-creator supports a large number of output types, and only some ofthese are currently available via lorax-composer. To add a new output type tolorax-composer a kickstart file needs to be added to ./share/composer/. Thename of the kickstart is what will be used by the /compose/types route, and thecompose_type field of the POST to start a compose. It also needs to havecode added to the pylorax.api.compose.compose_args() function. The_MAP entry in this function defines what lorax-composer will pass topylorax.installer.novirt_install() when it runs the compose. When thecompose is finished the output files need to be copied out of the builddirectory (/var/lib/lorax/composer/results/<UUID>/compose/),pylorax.api.compose.move_compose_results() handles this for each type.You should move them instead of copying to save space.
If the new output type does not have support in livemedia-creator it should beadded there first. This will make the output available to the widest number ofusers.
Example: Add partitioned disk support¶
Partitioned disk support is something that livemedia-creator already supportsvia the --make-disk cmdline argument. To add this to lorax-composer itneeds 3 things:
- A partitioned-disk.ks file in ./share/composer/
- A new entry in the _MAP in pylorax.api.compose.compose_args()
- Add a bit of code to pylorax.api.compose.move_compose_results() to move the disk image fromthe compose directory to the results directory.
The partitioned-disk.ks is pretty similar to the example minimal kickstartin ./docs/rhel7-minimal.ks. You should remove the url and repocommands, they will be added by the compose process. Make sure the bootloaderpackages are included in the %packages section at the end of the kickstart,and you will want to leave off the %end so that the compose can append thelist of packages from the blueprint.
The new _MAP entry should be a copy of one of the existing entries, but with make_disk setto True. Make sure that none of the other make_* options are True. The image_name iswhat the name of the final image will be.
move_compose_results() can be as simple as moving the output file intothe results directory, or it could do some post-processing on it. The end ofthe function should always clean up the ./compose/ directory, removing anyunneeded extra files. This is especially true for the live-iso since it producesthe contents of the iso as well as the boot.iso itself.
Package Sources¶
By default lorax-composer uses the host’s configured repositories. It copiesthe *.repo files from /etc/yum.repos.d/ into/var/lib/lorax/composer/repos.d/ at startup, these are immutable systemrepositories and cannot be deleted or changed. If you want to add additionalrepos you can put them into /var/lib/lorax/composer/repos.d/ or use the/api/v0/projects/source/* API routes to create them.
The new source can be added by doing a POST to the /api/v0/projects/source/newroute using JSON (with Content-Type header set to application/json) or TOML(with it set to text/x-toml). The format of the source looks like this (inTOML):
name = "custom-source-1"url = "https://url/path/to/repository/"type = "yum-baseurl"proxy = "https://proxy-url/"check_ssl = truecheck_gpg = truegpgkey_urls = ["https://url/path/to/gpg-key"]
The proxy and gpgkey_urls entries are optional. All of the others are required. The supportedtypes for the urls are:
- yum-baseurl is a URL to a yum repository.
- yum-mirrorlist is a URL for a mirrorlist.
- yum-metalink is a URL for a metalink.
If check_ssl is true the https certificates must be valid. If they are self-signed you can either setthis to false, or add your Certificate Authority to the host system.
If check_gpg is true the GPG key must either be installed on the host system, or gpgkey_urlsshould point to it.
You can edit an existing source (other than system sources), by doing a POST to the new routewith the new version of the source. It will overwrite the previous one.
A list of existing sources is available from /api/v0/projects/source/list, and detailed infoon a source can be retrieved with the /api/v0/projects/source/info/<source-name> route. By defaultit returns JSON but it can also return TOML if ?format=toml is added to the request.
Non-system sources can be deleted by doing a DELETE request to the/api/v0/projects/source/delete/<source-name> route.
The documentation for the source API routes can be found here
The configured sources are used for all blueprint depsolve operations, and for composing images.When adding additional sources you must make sure that the packages in the source do notconflict with any other package sources, otherwise depsolving will fail.
DVD ISO Package Source¶
In some situations the system may want to only use a DVD iso as the packagesource, not the repos from the network. lorax-composer and anacondaunderstand file:// URLs so you can mount an iso on the host, and replace thesystem repo files with a configuration file pointing to the DVD.
Stop the lorax-composer.service if it is running
Move the repo files in /etc/yum.repos.d/ someplace safe
Create a new iso.repo file in /etc/yum.repos.d/:
[iso]name=isobaseurl=file:///mnt/iso/enabled=1gpgcheck=1gpgkey=file:///mnt/iso/RPM-GPG-KEY-redhat-release
Remove all the cached repo files from /var/lib/lorax/composer/repos/
Restart the lorax-composer.service
Check the output of composer-cli status show for any output specific depsolve errors.For example, the DVD usually does not include grub2-efi-*-cdboot-* so the live-iso imagetype will not be available.
If you want to add the DVD source to the existing sources you can do that bymounting the iso and creating a source file to point to it as described in thePackage Sources documentation. In that case there is no need to remove the othersources from /etc/yum.repos.d/ or clear the cached repos.