diff options
author | Laurent Bercot <ska-skaware@skarnet.org> | 2017-05-22 22:35:47 +0000 |
---|---|---|
committer | Laurent Bercot <ska-skaware@skarnet.org> | 2017-05-22 22:35:47 +0000 |
commit | 903059dfb0dc520f0618062bbbafac4ebd8e29f8 (patch) | |
tree | 58d54a90a1dce411cbade9d26702c4a307c1518c /documentation/DEPENDENCIES-BUILD.md | |
parent | 3e9a6410bc6265fd322bedf4ac54dd85eea8e51e (diff) | |
download | lh-bootstrap-903059dfb0dc520f0618062bbbafac4ebd8e29f8.tar.xz |
Add index.html, rename documentation to doc, clean up
Diffstat (limited to 'documentation/DEPENDENCIES-BUILD.md')
-rw-r--r-- | documentation/DEPENDENCIES-BUILD.md | 175 |
1 files changed, 0 insertions, 175 deletions
diff --git a/documentation/DEPENDENCIES-BUILD.md b/documentation/DEPENDENCIES-BUILD.md deleted file mode 100644 index e1d97de..0000000 --- a/documentation/DEPENDENCIES-BUILD.md +++ /dev/null @@ -1,175 +0,0 @@ - -# lh-bootstrap: software built for the BUILD machine - -Laurent Bercot -2016-03-31 - - -This file documents the software installed and run on the BUILD -machine prior to building the HOST image. - -Please read the INTERNALS.md file first, for the general organization -of the build, and basic definitions. - - -## BUILD tools - -### Linux kernel headers - -Makefile directory: sub/kernel - -The Linux kernel is downloaded and will be configured and compiled -to boot a qemu image for the HOST. Since it will be downloaded -anyway, we reuse the source to process and install the kernel headers -for the BUILD. -Those kernel headers, coupled with the musl libc's headers, are -necessary to compile Linux-specific software such as util-linux and skarnet-org. - - -### musl libc - -Makefile directory: sub/musl - - We have no control over the BUILD's native compiler and libc. Most -likely, it's gcc and produces binaries that are dynamically linked -against the glibc - but we're not certain; we would like certainty, -even for the build tools. We do not want our tools' behaviour to -depend on external factors such as a misconfigured libc or dynamic -linker. - - So, we download the musl libc (which we would download for use in -the HOST anyway) and compile it for the BUILD. We then link all our -BUILD tools against it. - - -### skarnet.org packages - -Makefile directory: sub/skarnet.org - - The HOST uses s6-rc as its service manager. We provide a template -for the database in source format in `layout/rootfs/etc/s6-rc/source-base`; -this template is preprocessed and added to the rootfs at layout -installation time, at the beginning of the HOST build. - However, in order to boot, the HOST needs the database in compiled -form, not in source form: so we must run s6-rc-compile before the HOST -boots. Since the source and compiled formats are platform-independent, -we just run s6-rc-compile on the BUILD. Which means we need to compile -s6-rc for the BUILD, with the same settings that the HOST is using. -So we end up compiling most of the skarnet.org stack. - - Since we have to compile skalibs anyway, which is by far the heaviest -component of the stack, we also use the opportunity to compile -s6-portable-utils for the BUILD: the time spent compiling this package -is negligible once skalibs is built, and it contains -alternative tools that we use subsequently in the build, because their -behaviour is more predictible than the tools provided by the BUILD's -distribution. - - Note: since we need to mirror the HOST's layout for s6-rc-compile -to work properly, we compile the skarnet.org stack following the -slashpackage convention, with --enable-slashpackage. However, we -obviously don't install a slashpackage hierarchy on the BUILD's root -filesystem, we use the $(OUTPUT)/build-build staging directory. -The consequence is that skarnet.org binaries that exec other binaries -via slashpackage paths will not work. This is ok for our use since -the main tool we need is s6-rc-compile, which does not exec anything -else, but it's something to keep in mind. It's the reason why we do -not use s6-setuidgid even after building s6: we stick to the hackish -and inefficient bin/setuidgid script to drop privileges, because our -temporary installation of s6-setuidgid simply does not work. - - -#### skalibs - - The library which all other skarnet.org packages depend on. - - -#### execline - - The scripting language used by s6 and s6-rc. - - -#### s6 - - The supervision suite used by s6-rc. - - -#### s6-rc - - The service manager used by the HOST. We compile it for the BUILD in -order to use s6-rc-compile to compile the service database before -booting the HOST. - - -#### s6-portable-utils - - Some utilities are akin to POSIX tools, but will have reproducible behavior -no matter what distribution is used. We have had trouble with -differences across BUILD distributions, with some distributions -slightly deviating from the standard (looking at you, Ubuntu); using -our own tools is insurance against that. - - -### util-linux - -Makefile directory: sub/util-linux - - To make the qemu image, we need losetup -P, to set up a loopback -mount that supports partitions. But the -P option to losetup only -appear in latest versions of util-linux, and not all distributions -ship a recent enough version. (Looking at you, Ubuntu and Debian -stable.) - So we download and build util-linux. Except the util-linux -build system is a bloated plate of noodles, that can have a lot -of dependencies - in particular a dependency to ncurses, and we -DO NOT want to build ncurses if it can be avoided. Fortunately, -none of the tools we need require ncurses. So we end up building -those individual binaries from util-linux and avoid pulling in -the kitchen sink. - Currently, the binaries we build are: losetup, fdisk, mkswap, -mount, umount. This list can change as the package evolves; the -current list is described in the UTLX_PROGLIST variable definition -in the sub/util-linux/Makefile file. - - -### xz-utils - -Makefile directory: sub/xz - - xz-utils includes another compression library (liblzma), which -is also a dependency of kmod - actually, this is the one that -interests us. So we have to build the xz-utils package for -BUILD. - - -### kmod - -Makefile directory: sub/kmod - - Ah, kmod. - - We build the Linux kernel for HOST with module support, for -practicality. Modules are compressed, to save storage space. -Traditionally, there are compressed with gzip (and have extension -`.ko.gz`), but xz is generally a better compressor than gzip: -it decompresses faster and the compressed data is smaller. So -we use xz to compress the modules (extension `.ko.xz`). On the -HOST, we load the modules with busybox modprobe, which supports -both extensions. So far, so good. - - Except that xz support for kmod is relatively recent, and some -distributions insist on providing an ancient version of kmod, -which *does not* allows modules to be compressed with xz. -(And the kernel's build system does not report the error - the -modules silently fail to be installed, which makes diagnostic -fun!) - - So, we have to provide our own version of kmod. - - I have to say that kmod is the single worst package that appears -in this whole build. The software itself works, but the -build system is *extremely* buggy and requires several workarounds, -that have all been implemented in the Makefile. Please do not -attempt to "simplify" this Makefile by using "correct" configure -options and eliminating make variables: that will not work. - |