Getting BusyBox with It

Following up on the Building a Barebones Linux System post, today I will be adding BusyBox to make the system a smidge more usable.

What is this BusyBox?

BusyBox is a refers to itself as The Swiss Army Knife of Embedded Linux. It is a collection of common Unix tools living within a single binary that uses the magic of argv[0] to facilitate a lightweight implementation of common programs like sh, ls, cat and many more. It is extremely common in the embedded systems community because it is a well maintained open source project and can be built very small.

For my purposes, I will use BusyBox instead of going into a full Linux From Scratch route. This will give me a shell and a few other basic userspace tools needed to allow me to make some use of this bare bones system.



Automating the builds

At this point I have been dabbling with this bare bones setup long enough that I want some additional tooling to help manage rebuilding as necessary. There are many good tools out there for managing custom Linux builds, but for now I’d like to continue using custom tools to learn more of what it takes to build a Linux distro from square one.

I’ve made a repo on Github and a Makefile that will pull in the kernel, build it, and build my initramfs. It also has a runvm command that will help me test my builds quickly.

KERNEL_VERSION=4.13.3
KERNEL_DIRECTORY=linux-$(KERNEL_VERSION)
KERNEL_ARCHIVE=$(KERNEL_DIRECTORY).tar.xz
KERNEL_URL=https://cdn.kernel.org/pub/linux/kernel/v4.x/$(KERNEL_ARCHIVE)


all: vmlinuz initramfs


# Kernel build targets
vmlinuz: $(KERNEL_DIRECTORY)
  cd $(KERNEL_DIRECTORY) && make defconfig && make -j`nproc`
  cp $(KERNEL_DIRECTORY)/arch/x86_64/boot/bzImage vmlinuz

$(KERNEL_DIRECTORY):
  wget $(KERNEL_URL)
  tar xf $(KERNEL_ARCHIVE)


# Initramfs build targets
initramfs: initfs initfs/init
  cd initfs/ && find . | cpio -o --format=newc > ../initramfs

initfs/init: initfs init.c
  gcc -o initfs/init -static init.c

initfs:
  mkdir -p initfs/bin


# Utility targets
runvm: vmlinuz initramfs
  qemu-system-x86_64 -m 2048 -kernel vmlinuz -initrd initramfs

clean:
  rm -rf vmlinuz $(KERNEL_DIRECTORY) $(KERNEL_ARCHIVE)

 

Configuring and Building BusyBox

BusyBox uses buildroot so its configuration and build process are very similar to the kernel itself. After downloading and extracting the source I’ll use make menuconfig to use the ncurses based config editor.

I’ll be leaving most settings at their default, but will configure the build to generate a static binary since I don’t yet have a libc implementation to link against.

Configuring and Building BusyBox

With BusyBox built, I’ll add it to my initfs directory so that my Makefile includes it in my initramfs.

Copying busybox binary to my initfs/bin directory

The busybox binary will act differently based on what it is called as, if we renamed it to sh it would act as a shell, or if named ls it will list our files and such. To allow it to serve all of these functions I’ll check what commands my build implements and create symlinks for each program it provides.

Creating the symlinks for BusyBox

Switching the Build to BusyBox

With my process for building BusyBox roughly figured out, I’ll save a copy of my BusyBox config, so that future builds will also be static linked. I’ll also include a small shell script that builds my symlinks.

cd initfs/bin
for i in `./busybox --list`
do
  ln -s busybox $i
done

Now that I have BusyBox in my initramfs, I’ll switch my init process from my custom binary to a shell script that will forever spawn a shell to use.

#!/bin/sh

while true
do
  sh
  sleep 1
done

With this script in place I’ll update my Makefile and repo to include the changes I’ve made here.

And now we test!

Testing the build in QEMU

There are a few deficiencies in here still, e.g. ps won’t work because I have no procfs mounted and sh will complain about being unable to access tty.

To get a few more things to run sanely, I’ll add /dev, /proc and /sys to my initramfs and modify my init script to mount the procfs and sysfs. I’ll also run mdev -s which is provided by BusyBox to populate /dev and /sys. Lastly, I’ll launch my shell with setsid cttyhack sh so that job control is enabled.

With all that setup, the system is substantially more usable! I’ll push these changes to my repo and call it a wrap. If you have any questions or feedback please let me know in the comments 😀

Save

Save

Save

Barebones Linux on Barreleye G1

Following up on the previous article, this post will be building a similar barebones Linux for the POWER8-based OpenPOWER Barreleye system.

For this round, I will be building the system on a freshly kicked x86_64 Ubuntu 16.04 VM to capture all dependencies needed to follow the same steps.



Cross Compiling Linux

To get things started I will pull in the various dependencies needed to build the system.

sudo apt install -y build-essential gcc-powerpc64le-linux-gnu libssl-dev bc

With these dependencies in place, I’ll create a new directory to work out of and pull down the kernel the same way as the last post. A minor difference is today the latest stable version is 4.13.1, so I’ll be rolling with that version.

mkdir barebones-ppc64le
cd barebones-ppc64le/
wget https://cdn.kernel.org/pub/linux/kernel/v4.x/linux-4.13.1.tar.xz
tar xf linux-4.13.1.tar.xz
cd linux-4.13.1/

Next I setup some environment variables to prepare for cross compilation.

export ARCH=powerpc
export CROSS_COMPILE=/usr/bin/powerpc64le-linux-gnu-

The next big step is to configure the kernel. For this platform, I have not yet had success using one of the default configs that comes with the kernel source tree. The good news is that the OpenPOWER firmware of this platform is Linux based and open source. The kernel configuration from the firmware build repo is a known good configuration for this exact system, so that gives me assurance it will work pretty well.

I’ll download the skiroot kernel config, run the kernels make oldconfig to get the kernel build happy with the configuration (piping yes '' to make to accept the default answers to its many questions), and build the kernel using the number of threads returned to me by nproc.

wget https://raw.githubusercontent.com/open-power/op-build/master/openpower/configs/linux/skiroot_defconfig -O .config
yes '' | make oldconfig
make -j`nproc`
cp vmlinux ..
cd ..

Building initramfs and Testing

Building the initramfs is also nearly identical to the previous post, just using the cross compiler instead of the default. This is the program that will be my init process.

#include <stdio.h>
#include <unistd.h>
int main() {
  while (1) {
    printf("Hello from the land of Barreleye\n");
    sleep(1);
  }
}

And the steps I used to compile and pack into my initramfs

powerpc64le-linux-gnu-gcc -o init -static test.c
echo init | cpio -o --format=newc > initramfs

With that all done, I will upload it to a webserver I have in our lab and boot up my Barreleye system to Petitboot, the firmware stacks primary interface. I’ll exit to a shell, wget the kernel and initramfs and kexec load and execute!

Since I’m connecting into the system from home to the BMC console the interface is a wee bit slow to respond to input, but it does the trick.

Testing my build on a Barreleye system

All looks good! Please feel free to comment or leave any questions you may have.

Save

Building a Barebones Linux System

In this post, I will describe the process I’ve used to create a very minimal Linux build. It is minimal for the sake of learning, to get a clearer picture of how barebones a operating system could be put together. The build will consist of an operating system kernel and an initial ram disk

I will be using Ubuntu 16.04 on my laptop as my build box, and use QEMU VMs as my early test systems. If you’d like more information on building a linux kernel there is a decent tutorial on kernelnewbies as well as the kernel building section of the Gentoo handbook.



The Linux Kernel

It can be difficult to see the separation of what is Linux kernel and what is the Linux distribution. The Linux Kernel itself is there primarily to facilitate access to hardware and provide a safe haven for processes to live and breathe. The kernel’s user interface is its API. It provides a foundation for all other software to allow storing files, using networks, render graphics, handle keyboard input and so on. On top of the kernel your terminal, your login process, your window manager, your browser, etc. are usually provided by your distribution.

Obtaining and building the kernel is very simple, though configuring the kernel can be a little trickier depending on your needs. For the sake of this excersize the default configurations for an x86_64 system will do just fine.

First off, I’ll create a new directory to work out of and get started. As of today, the latest linux kernel at the top of kernel.org is 4.13, I will download this latest tar file, extract it and jump into the extracted directory.

Downloading the kernel

Now that I have the kernel source files, I’ll use make defconfig to configure the defaults for a standard x86_64 machine and will do make -j4 to compile the kernel using all 4 threads my laptop has to offer.

Starting the kernel build

Even with the extra threads, it takes about 8 minutes to compile on my machine. It’s not the simplest kernel as there are many default features we could disable to trim down the kernel size and compile time but I’m not worried about that for this build. Once it’s complete I copy the bzImage file that was generated into my barebones parent directory.

Kernel build finishing

Init process and initramfs

Typically when a computer boots, the host firmware will look for a bootloader on a bootable device. The bootloader will load the kernel and pass control of the system to it. The kernel gets control, does it’s own startup shenanigans and looks for a program to run as the primary (init) userspace process (PID 1)

In most Linux distributions, this init process can be a SysV style init process, SystemD, Upstart or Upstart. In this case, it’s going to be a looping hello world program. I compile it with -static so that libraries are staticly linked

Compiling a simple hello world looping program

The next step is to build the initramfs, the initial file system that will go in ram for the kernel to interact with. To do this, I will pipe a newline separated list of files to pack to cpio. I only have one file so I’ll just echo that to it. cpio will output the archive and I’ll store that to a file.

Building an uncompressed initramfs with a single file

Testing the new build in QEMU

I’ll be using qemu to test the new build. The simplest way I could boot the VM would be to use qemu-system-x86_64 -kernel vmlinuz -initrd initramfs, but to run this within my terminal I’ll also add -append console=ttyS0 -nographic.



Testing the kernel in QEMU

It boots fast and my hello world endlessly spits out its message! My barebones OS works! I may eventually use this to build a few utility images to network boot but for now this will do as a base for more tinkering.