Folks have expressed a bit of interest in some of the lower level of what I’m doing with Raspberry Pi OS build stuff. My Tablet now has over 50 links open and I need to get some of them closed. Put those together, you get a posting that preserves the links so I can find them again, and informs folks what kind of stuff I’m looking at for the Build Monster set up and Upstream Candidates.
There won’t be a lot of analysis, that will come later. But each of these links is something I’m digging into to see what I think, and what I can extract to use in MyBuild. Note that these are in no particular order (just in order of opening on my tablet…gathered into groups)
Gentoo
There is a straight forward process for putting Gentoo onto the Raspberry Pi:
https://wiki.gentoo.org/wiki/Raspberry_Pi
The Raspberry Pi is series of single-board computers based on ARM devices. Raspberry Pi 1 is based on BCM2835 SoC and features a single core CPU (ARMv6), with 512 MB RAM. Earlier models had 256 MB RAM. Raspberry Pi 2 is based on BCM2836 SoC and features a quad-core CPU (ARMv7) and 1 GB RAM. The model 1 uses an SD card for storage but the model 2 uses a microSD. All versions have a VideoCore IV GPU. This document describes how to install Gentoo on the Raspberry Pi.
Contents
1 Preparing the SD card
1.1 Partitioning
1.2 Boot partition
1.3 Stage 3
1.4 Stage 4
1.5 Portage tree
2 Compiling the kernel
2.1 crossdev
2.2 Kernel and modules
2.2.1 Manual compilation
2.2.2 Using genkernel
2.2.3 Installing kernel image
3 Tips and tricks
4 Troubleshooting
5 See also
6 External resources
7 Books
8 References
Reading it, there are places where it is a bit down in the bit weeds and applying other folks tools would speed things up, in particular prepping the SD card and such. Just let NOOBs do it, or use gparted and a nice graphical interface… It isn’t like you can’t install Fedora of Debian on a chip in an hour and have those tools available.
Gentoo has a rather nice Cross Building for the Pi page here:
https://wiki.gentoo.org/wiki/Raspberry_Pi_Cross_building
Building almost anything on the Raspberry Pi takes a very, very long time – especially when a lot of dependencies are involved.
Fortunately, it is possible to offload much of the heavy lifting to a more powerful system (such as your main Gentoo desktop/server) using distcc and crossdev (though this will only work for packages that are written in language that require compiling).
It is more oriented toward the original Pi, from what I’ve seen so far.
Gentoo has ‘flexibility’ as a basic goal, since many folks use it as a base for embedded systems development. So lots of cross development tools letting you compile on your PC and run on the SOC-duJour… They also support both gligc for the big-fully-functional libraries and uClibc for the “small mostly for embedded systems maybe bits not there” set. They have “hardened” versions too. Also, they let you choose between systemD and OpenRC. Nice. Very nice.
It is technically rich and technically challenging, aimed at hard core tech types. But is IMHO the best candidate for “upstream” for the Raspberry Pi since ARM chips are widely used in embedded systems, it is tuned for that level of hardware speed, is not bloat and SystemD afflicted, and is VERY security aware. (You can’t just push a patch to the pkg archive when a system crack is found… those 100,000 wifi hotspots you sold are going to sit exposed, potentially forever, so more effort goes into up-front security). The release is distributed as source, and you “roll your own” which is good for me, but means a day spent compiling things for most folks. It expects you are a system builder.
They have a pointer to using a cross compiler with “emerge” that looks to me to be a tool local to their distribution of Linux. For Gentoo I’d likely use this with a cluster of distcc cores, or if that craps on me, my largest PC as cross development tool. BTW, folks who have done devo before tend to think in terms of alternative paths as that often ends up being what you do, like it or not. “Be like water, still, but flowing, flow past problems and rejoin on the other side”…
http://dev.gentoo.org/~vapier/CROSS-COMPILE-HOWTO
HOWTO BUILD A CROSS COMPILER WITH EMERGE !
————————————————————
Before you read any further, you should try using ‘crossdev’
in portage. You’re less likely to screw something up that
way. Simply do `emerge ‘>=crossdev-0.9’` and then execute
`crossdev –help`.The insturctions below are for those who wish to learn
more about the details of how crossdev works wrt portage.
You could probably give two shits, so just use crossdev !————————————————————
————-
PACKAGE NOTES
————-– binutils
Only binutils versions that ‘inherit toolchain-binutils’
will work. The host system should also being using one
of these versions or you’ll probably break your system.
– linux-headers
Any version that has ‘inherit kernel-2’ should work.
– gcc
Only gcc versions that ‘inherit toolchain’ will work.
Supported versions:
3.3.6+
3.4.6+
4.0.3+
4.1.1+
Anything else you’re on your own.
– glibc
Supported versions:
2.3.6-r4+
2.4-r3+
Anything else you’re on your own.
etc. etc. for pages more… You read it all, then make a best guess as to what path will work best for you, hop in the river, shoot the rapids, and try not to drown while you cope with it until Ship Date comes… then you hand it to Q.A. and start all over again…
And also some pointers on using distcc
https://wiki.gentoo.org/wiki/Distcc/Cross-Compiling
1 Cross-compiling with distcc
1.1 Introduction
1.2 Emerge the needed utilities
1.3 Arch-specific notes
1.3.1 Intel x86 subarchitectures
1.3.2 SPARC
1.4 Configuring distcc to cross-compile correctly
1.5 How this works
1.6 Troubleshooting
I note in particular they reference the particulars of using Arch as a Gentoo cross dev world with distcc… just what I was thinking of doing.
Arch-specific notes
Note
Obtain the architecture name by looking at the compile target’s CHOST variable within /etc/make.conf. When mangling the architecture name for the crossdev -t option, crossdev will merrily guess and install tools using the mangled architecture name for folder names within /usr (ie. /usr/i686-pc-linux-gnu/, /usr/i686-linux-gnu/, …). To resolve this, specify each mangled architecture/folder name to crossdev –clean for uninstalling, or manually remove the folders from the system.
OK, so the ‘crossdev’ is not very pristine about where it finds things and will steal Arch specific stuff and use it if you are not careful, so be careful. Got it.
The really good thing is that a lot of folks have already made ARM work with Gentoo, and you can download a build environment stage pretty easily. (Integrating it with the Pi specific boot process is a bit more…)
http://distfiles.gentoo.org/releases/arm/autobuilds/current-stage3-armv7a_hardfp/
[PARENTDIR] Parent Directory –
[ ] stage3-armv7a-20160325.tar.bz2 2016-03-26 14:21 218M
[ ] stage3-armv7a-20160325.tar.bz2.CONTENTS 2016-03-26 14:21 4.6M
[ ] stage3-armv7a-20160325.tar.bz2.DIGESTS 2016-03-26 14:21 724
[TXT] stage3-armv7a-20160325.tar.bz2.DIGESTS.asc 2016-03-26 19:32 1.6K
[ ] stage3-armv7a_hardfp-20160325.tar.bz2 2016-03-26 04:23 218M
[ ] stage3-armv7a_hardfp-20160325.tar.bz2.CONTENTS 2016-03-26 04:23 4.6M
[ ] stage3-armv7a_hardfp-20160325.tar.bz2.DIGESTS 2016-03-26 04:23 752
[TXT] stage3-armv7a_hardfp-20160325.tar.bz2.DIGESTS.asc 2016-03-26 07:32 1.6K
In theory, slide one of those into a NOOBs chip and you have a good starting point… in theory…
Linux From Scratch LFS
This is a project, more than a release. It is intended to teach how to roll your own Linux Distro. All source is complied locally, so it takes a while to build (figure a few days as you learn bits too). In the end, you have a low feature Linux with a command prompt. THEN you can advance to BLFS Beyond Linux From Scratch and do more to eventually get things like an X window system and maybe even a browser. There are several “forks” (variations) with themes like the BLFS folks, CLFS for cross dev embedded oriented style, and more. I’ll likely build one just for the experience and knocking rust off of some of the old “compile a kernel” and “compile the compiler” tricks…
http://www.linuxfromscratch.org/
Welcome to Linux From Scratch!
Linux From Scratch (LFS) is a project that provides you with step-by-step instructions for building your own custom Linux system, entirely from source code.
Currently, the Linux From Scratch organization consists of the following subprojects:
LFS :: Linux From Scratch is the main book, the base from which all other projects are derived.
BLFS :: Beyond Linux From Scratch helps you extend your finished LFS installation into a more customized and usable system.
ALFS :: Automated Linux From Scratch provides tools for automating and managing LFS and BLFS builds.
CLFS :: Cross Linux From Scratch provides the means to cross-compile an LFS system on many types of systems.
HLFS :: Hardened Linux From Scratch focuses on building an LFS system with heightened security.
Hints :: The Hints project is a collection of documents that explain how to enhance your LFS system in ways that are not included in the LFS or BLFS books.
LiveCD :: The LiveCD project provides a CD that is useful as an LFS build host or as a general rescue CD.
Patches :: The Patches project serves as a central repository for all patches useful to an LFS user.
Note in particular the “ALFS” project that has a script builder. It looks at their book and extracts from it a script that is likely to build the system automatically. Nice.
Also note the “HLFS” project that does the hardening. And the “LiveCD” folks that have done the work of making it a LiveCD with hgardware probe and RAMfs and such.
Essentially, most or all of the bits are there for my goals, just needs assembly.
(Some assembly required ;-)
I’ve looked at their books and they are very well done. Fairly easy to read, aimed at someone with a little Linux Experience, but very handholdy and cookbook. Nice, very very nice.
Their book is available for free download here:
http://www.linuxfromscratch.org/lfs/downloads/stable/
It gives a very good idea what level of work is involved in a ‘roll your own’ operation.
Specs and what you need to pay attention to for a build system:
http://www.linuxfromscratch.org/lfs/view/development/prologue/hostreqs.html
Note that the particular minor release numbers of all the parts individually matter. Libraries, compilers, kernel et. al. must be matched. (This is important at this level of build, not so much for compiling “Hello World” in user-land…)
Your host system should have the following software with the minimum versions indicated. This should not be an issue for most modern Linux distributions. Also note that many distributions will place software headers into separate packages, often in the form of “-devel” or “-dev”. Be sure to install those if your distribution provides them.
Earlier versions of the listed software packages may work, but has not been tested.
Bash-3.2 (/bin/sh should be a symbolic or hard link to bash)
Binutils-2.17 (Versions greater than 2.26 are not recommended as they have not been tested)
Bison-2.3 (/usr/bin/yacc should be a link to bison or small script that executes bison)
Bzip2-1.0.4
Coreutils-6.9
Diffutils-2.8.1
Findutils-4.2.31
Gawk-4.0.1 (/usr/bin/awk should be a link to gawk)
GCC-4.7 including the C++ compiler, g++ (Versions greater than 6.1.0 are not recommended as they have not been tested)
Glibc-2.11 (Versions greater than 2.23 are not recommended as they have not been tested)
Grep-2.5.1a
Gzip-1.3.12
Linux Kernel-2.6.32
The reason for the kernel version requirement is that we specify that version when building glibc in Chapter 6 at the recommendation of the developers. It is also required by udev.
If the host kernel is earlier than 2.6.32 you will need to replace the kernel with a more up to date version. There are two ways you can go about this. First, see if your Linux vendor provides a 2.6.32 or later kernel package. If so, you may wish to install it. If your vendor doesn’t offer an acceptable kernel package, or you would prefer not to install it, you can compile a kernel yourself. Instructions for compiling the kernel and configuring the boot loader (assuming the host uses GRUB) are located in Chapter 8.
M4-1.4.10
Make-3.81
Patch-2.5.4
Perl-5.8.8
Sed-4.1.5
Tar-1.22
Texinfo-4.7
Xz-5.0.0
Etc. etc. and so it goes. Building a system is a very picky and very precise low level operation. Get sloppy, you are in the land of unexplained bugs for many days and many nights…
Then, if you decide to go “off the reservation” (to quote Hillary per men… yet that isn’t racist and sexist?… sheesh…) and do distributed compilation and development, it gets worse, but they have some pointers:
http://www.linuxfromscratch.org/hints/downloads/files/distributed_compiling.txt
AUTHOR: Cedric de Wijs cedric.de.wijssofthomenet
DATE: 2004-04-09
LICENSE: GNU Public Licence
SYNOPSIS: How to install distcc –a free distributed C/C++ compiler system
DESCRIPTION:
TODO: put some usefull words hereATTACHMENTS:
The distcc package, available from:
http://distcc.samba.org/ftp/distcc/distcc-2.12.1.tar.bz2 (317,8kB)PREREQUISITES:
This hint expect you to have two working lfs systems with the same version of gcc, binutils and glibc, and functional networkingHINT:
=================
TABLE OF CONTENTS
=================1 Introduction
2 Required Files
3 Installation server
4 Installation client
5 Usage
6 Pitfalls
7 Recommended reading===============
1 Introduction
===============Compiling sourcecode can take a long time. This hint describes how to install distcc, a tool that can be used to distribute the compiling across multiple machines in a network.
The machine you sit behind and issue the compile from is the client.
The machines who are doing the work are the servers.
Every machine can contain both a server and a clientThis text is based upon the documentation of distcc, written by Martin Pool
Estimated build time: 0.20 SBU
Estimated required disk space: 5.4 MB
and so on for pages more…
The Really Good News? Someone has already ported LFS to the Raspberry Pi, so you know where there be dragons:
http://www.intestinate.com/pilfs/guide.html
Build Guide
Linux From Scratch on the Raspberry PiThis guide will take you through the Pi specific steps of an LFS build.
About The GuideWe’re going to work our way through the development version of LFS, while paying extra attention to the steps where the instructions differ for the Pi. It is therefore recommended that you keep the LFS book open in another browser tab. You’ll find a pointer to the corresponding section of the LFS book at each step. The LFS book steps that are not mentioned can be assumed to work on the Pi exactly as printed.
This guide was tested and verified on an 8GB Kingston SD card with 2013-02-09-wheezy-raspbian as the host system but most Linux distributions should be able to work as a host platform for your LFS build as long as the required tools can be added. The minimum SD card size is probably 4GB. If you use the PiLFS base image, you’re all set and you save a few hours on the total build time with more free space for your build compared to Raspbian.
Grand total build time is approximately 60 hours on RPi 1 and 12 hours on RPi 2.
It is basically the “diffs” from the LFS book, so you need and use both. Figure a nice weekend project on a Pi Model 2 or 3. I think there are opportunities to shortcut their process. For example, using NOOBs or gpartd on Fedora to lay out the SD card. Still, it is very nice to have a map to the road ahead.
Odds & Ends
For anyone interested in a general overview of the software build process in a Unix / Linux world, and not wanting to be dependent on binary blobs via a package manager, here’s a good overview.:
http://www.tldp.org/HOWTO/Software-Building-HOWTO.html
1. Introduction
Many software packages for the various flavors of UNIX and Linux come as compressed archives of source files. The same package may be “built” to run on different target machines, and this saves the author of the software from having to produce multiple versions. A single distribution of a software package may thus end up running, in various incarnations, on an Intel box, a DEC Alpha, a RISC workstation, or even a mainframe. Unfortunately, this puts the responsibility of actually “building” and installing the software on the end user, the de facto “system administrator”, the fellow sitting at the keyboard — you. Take heart, though, the process is not nearly as terrifying or mysterious as it seems, as this guide will demonstrate.
As an “And oh yeah, it might work too”; FreeBSD has Arm6 images for the first Pi, among others, so a v7 image ought to be buildable too.
ftp://ftp.freebsd.org/pub/FreeBSD/snapshots/arm/armv6/ISO-IMAGES/11.0/
This page:
https://wiki.freebsd.org/FreeBSD/arm/Raspberry%20Pi%202%20image
says how…
Building an image for Raspberry Pi 2
Update: Raspberry Pi 2 is supported by the Crochet build tool.
Build and install the u-boot port in sysutils/u-boot-rpi2 (or install the package)
Create the image file (and setup the partitions):
A couple of times I’ve said I’d just “use NOOBs or Berryboot”… this link shows how to add an OS to a BerryBoot SD chip:
http://berryterminal.com/doku.php/berryboot/adding_custom_distributions
Adding your own custom operating systems to the menu
You can add your own extra operating systems to the Berryboot menu. However this requires that you convert your file system image to SquashFS format first.
Most Raspberry Pi operating system images are disk images containing two partitions. A FAT partition with the boot loader and kernel files, and a second ext4 partition with everything else. We are interested in the second partition.
With a regular Linux desktop computer that has kpartx and mksquashfs installed, you can convert the second partition to SquashFS like this:
$ sudo kpartx -av image_you_want_to_convert.img add map loop0p1 (252:5): 0 117187 linear /dev/loop0 1 add map loop0p2 (252:6): 0 3493888 linear /dev/loop0 118784 $ sudo mount /dev/mapper/loop0p2 /mnt $ sudo sed -i 's/^\/dev\/mmcblk/#\0/g' /mnt/etc/fstab $ sudo mksquashfs /mnt converted_image_for_berryboot.img -comp lzo -e lib/modules $ sudo umount /mnt $ sudo kpartx -d image_you_want_to_convert.imgNotes:
If kpartx reports it created a mapping different than loop0p2 (e.g. loop4p2) mount that instead. This can happen if loop0 is already in use by something else on the system.
We are excluding /lib/modules from the image, because the kernel modules shipped with Berryboot are used instead, and shared with all distributions.
Some older versions of mksquashfs do not support the ”-comp lzo” option. You can leave it out to let it use gzip compression instead. Advantage of LZO is that it is faster to uncompress, which is a big plus on slow ARM devices, and therefore preferred. This does come at a cost of reduced compression ratio (LZO images are larger than gzip ones).Put your SquashFS formatted image on a USB stick, go to the “Operating system installer”, hold down your mouse button over “Add OS” and select “Install from USB stick” If your image prefers to have a certain memory split use the extension .img128 .img192, .img224 or .img240 instead of .img.
Closing Note
Now it may well look like a Royal PITA to read all that stuff, but it is WAAAYYY easier to read it, think some, and find the best path through it, than to just “pick one and start coding.”. Often you can saves days or weeks by up front think time and a better choice.
On one of my systems I’d found a neat page from IIRC the UK, where someone laid out the path to an easier Gentoo install, using NOOBS for the SD formatting and boot code, and then using a Raspian kernel to boot (skipping that whole build a kernel phase)and then layering userland on top from Gentoo. Ah, found it:
Install Gentoo on Raspberry Pi 2 / 3 + USB Hardrive
This method uses a NOOBS install of Raspbian as the host to install Gentoo onto a USB Hardrive. It is quick & simple, providing a working Gentoo install with the least pain in the shortest time possible.
Please read ALL the instructions before attempting this installThis install method leaves NOOBS and Raspbian on the SD Card as the ultimate Rescue Disc (dual boot).
The install is quick and easy, using the stable Raspberry Pi Foundation Kernel off the SD-Card. No Kernel Compile required.
You’ll also end up with a machine theoretically fractionally faster because it everything will be compiled for ARMv7a_hardFP unlike other distros which are all compiled for the older ARMv6. I also provide an optional Stage-3 compiled for NEON vpf4 to enable all Hardware Acceleration, current-armv7a_neonvfpv4_hardfp.
What follows is How I Did It …. Every instruction has been copy pasted directly from my install. I then re-installed a couple of times following and these instructions to check they are “correct”. If you spot mistakes or can suggest improvements, please contact me.
Over View
The install falls roughly into 3 categories or stages.
Booting Raspbian Desktop to partition & format the USB Hardive
Download Base Files System, portage Package Stuff
chroot to sort the configs, such as set the keyboard, date, and such like.Ingredients
Rapsberry Pi 2 / 3
USB Hardrive
8Gb Micro SD card with NOOBSIt is trivial to purchase a suitable SD Card with NOOBS pre-installed. It is also wise, as not all SD Cards are created equal. This method demands NOOBS, as we will use at as an effective replacement for Grub/LILO.
Since very SanDisk, Kingston, and Samsung SD card I’ve bought has worked fine, and it takes about a minute to toss a NOOBS image onto a chip, I’m happy to do that. The rest of it is an interesting approach to using NOOBs to lever Gentoo. So that’s what I’m going to do next. It looks like they have a pointer to a NEON aware version too. I also like that they address putting things to an external USB drive. For a Daily Driver that alone could speed up things as data writes to SD card can be very slow. Since web pages cache a lot of stuff, that matters. Once I have this tested, I’ll post a report on it.
With that, off to dinner ;-)
Musings from the Chiefio 