Raspberry Pi chroot

So, let’s imagine you’re a happy owner of Raspberry Pi, you either just unpacked it, or may be used it for some time.
And for some reason you’ve decided to install not just your favourite binary distribution, but mighty source-based Gentoo linux.
First of all checkout Gentoo wiki pages dedicated to Raspberry Pi: [1], [2].
Also checkout Beyond Linux From Scratch on the Raspberry Pi as it also covers the topics we discuss here.

While Gentoo is already a lot about utilizing much of your processor power and I/O bandwidth for the compiling packages, Gentoo on a Raspberry Pi faces a real bottleneck when it comes on compiling on the same machine.

There are several ways to speed the things up.

  1. Hardware-related: speed up the RPi itself, i.e. build RPi build farm, buy smth else overclock it.
  2. Software related:
    • Distributed compilation: e.g. cross-compilation with distcc.
    • Building binary packages inside VM: qemu-system-arm, for QEMU+RPi initial configuration see [1], [2] (in Russian).
    • Building binary packages inside special environment: chroot + static qemu-system-arm.

At the time of writing the last option is quite poorly covered in gentoo wiki, however there is some info on this topic in archlinux wiki: [1], [2].
Here I try to extend the above mentioned links to Gentoo distribution.

Host configuration

Our running kernel should have kernel support for misc binaries (checkout zgrep CONFIG_BINFMT_MISC /proc/config.gz, it should return either y or m).

We need app-emulation/qemu package built with static-user and qemu_user_targets_arm USE flags.
Other flags defined by default should be sufficient.

Then we need to mount the binary format handler and register our format with the kernel via the procfs.

# [ -d /proc/sys/fs/binfmt_misc ] || modprobe binfmt_misc
# [ -f /proc/sys/fs/binfmt_misc/register ] || mount binfmt_misc -t binfmt_misc /proc/sys/fs/binfmt_misc
# /etc/init.d/qemu-binfmt start

Host configuration done.

Preparing the chroot environment

For the chroot environment one can use either stage 3 or stage 4 gentoo system or a disk image (such as Raspbian, or dd copy of SD card).
Links to the arm stage 3 tarballs: [1], [2].
For RPi1 we need armv6j-hardfp, for RPi2 – armv7a-hardfp.

# mkdir arm-chroot && cd arm-chroot
# tar -xvf stage3-arm-date.tar.bz2 #change the tarball name accordingly

Working with dd-image is slightly more tricky.

# losetup /dev/loop0 2015-04-24-nessensis.img -vP

where -P forces kernel to scan partition table on newly created loop device.
This command will create several block loop devices for each partition in the image: /dev/loop0p1, /dev/loop0p2, etc…
Typically RPi image consists of 2 partitions: /boot and /, the one we are going to chroot in.

# mkdir arm-chroot
# mount /dev/loop0p2 arm-chroot -o rw -o exec && cd arm-chroot

Now, we are in arm-chroot directory.
Prior to the chrooting, we need to mount certain filesystems.

# cp /etc/resolv.conf ./etc
# mount -t proc /proc proc -v
# mount --rbind /dev dev -v
# mount --rbind /sys sys

Optionally, we may also bind chroot’s portage to our system’s /usr/portage

# mkdir usr/portage && mount --rbind /usr/portage ./usr/portage -v

Copying the statically built qemu-arm cpu emulator to chroot environment and finally chrooting there.

# cp /usr/bin/qemu-arm ./usr/bin/qemu-arm
# chroot . /bin/bash --login

Now we’re in chroot and can actually start building packages!
Don’t forget adding FEATURES="buildpkg ${FEATURES}" to make.conf.
There are also several recent posts on problems with qemu chroot: [1], [2].

After we finish using it, leave the shell and umount previously mounted points.

As a closing remark, I’d like to mention that the cpu emulation is still not very fast.
I’ll publish benchmarks comparing chroot, VM and distcc soon.


Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s