My Magnificent Bastard System

After a few days of banging my head on the wall of an amrhf (32 bit) Devuan 2.0 release that just Did Not Run on my Raspberry Pi M3, and finding the FireFox on the arm64 (64 bit) Devuan 2.0 release just painfully slow, over the last 24 hours “I’ve created a monster!” ;-)

Essentially, I took a page from the first days of 64 Bit arm systems when the Linux kernel was new and userland was basically not ported to 64 bit.

I took the 64 bit install, and shoved it onto a mini-SD card. Then I made another partition where I could copy all of the / (root) file system of the armhf 32 bit version. Essentially, you keep all the stuff on the MSDOS partition from the 64 bit install (kernel, cmdline.txt config… etc.) and then change the cmdline.txt so it points to the partition where you un-tarred the 32 bit userland.

The one “tricky bit” is that kernel modules (loaded for things like network device drivers) are located in /lib/modules and you can’t run 32 bit modules in a 64 bit kernel. So you toss the 32 bit and copy in the 64 bit. Then you can do your first boot.

That’s basically it. Runs a champ!

Now I also decided to complexify things mightily while I was at it. At first I did the install into an 8 GB chip, then decided it wasn’t big enough for 2 whole systems, so moved to a 16 GB chip, but only AFTER having installed both, so I couldn’t effectively make the desired file system larger… which resulted in my making 3 more file systems on the chip and moving /var /usr and my home directory onto them.

Yeah, a bit messy, but lets me see just how big each one is with a df command AND sets me up for moving them onto a hard disk mounted partition OR using a squashfs file based file system for /usr to make it resistant to changes. (Once I’m done changing it, that is!) Just for kicks, I also made the added /var /usr and /home/chiefio partitions as xfs file systems. (So you must apt-get install xfsprogs on the system where you grab a copy of userland so it will be there at boot time…)

Here’s the final df output:

chiefio@devuan:~$   df
Filesystem     1K-blocks    Used Available Use% Mounted on
/dev/root        1743136  211904   1441020  13% /
devtmpfs          438328       0    438328   0% /dev
tmpfs              88764    1596     87168   2% /run
tmpfs               5120       4      5116   1% /run/lock
tmpfs             177520       0    177520   0% /run/shm
/dev/mmcblk0p1    130798   35754     95044  28% /boot
/dev/mmcblk0p3   4062912 2854276    998924  75% /arm64
/dev/mmcblk0p5   3135488  596576   2538912  20% /var
/dev/mmcblk0p6   4184064 1692296   2491768  41% /usr
/dev/mmcblk0p7   2073600  574264   1499336  28% /home/chiefio
tmpfs             443800       0    443800   0% /sys/fs/cgroup
tmpfs              88760       8     88752   1% /run/user/0
tmpfs              88760      16     88744   1% /run/user/1000

The /dev/root file system is the /dev/mmcblk0p2 slice and that is where the basic armhf file system is installed. I put the arm64 root in /arm64. It isn’t used, I just left it there for reference. Also, by changing the cmdline.txt file I still could boot in full 64 bit mode.

Here’s the cmdline.txt:

dwc_otg.fiq_fix_enable=2 console=ttyAMA0,115200 kgdboc=ttyAMA0,115200 console=tty1 root=/dev/mmcblk0p2 rootfstype=ext3 rootwait rootflags=noload net.ifnames=0

Note that I’ve made the file systems ext3 (instead of the default rootfstype=ext4) and you just change that root=/dev/mmcblk0p[2-3] to either 2 or 3 to change which userland boots.

When you install more software with apt-get, it caches the new programs .deb package in /var and then the actual install happens into /usr (usually). So both of those are where the growth happens as you fill out a system with all the “stuff” you want in it. When done configuring: /usr is relatively static (as all the ‘variable’ stuff was moved to /var some decades back). That means /usr can be mounted read only or even as a squashf file system on static finished configurations. (A minor added security measure).

Then /boot, the mmcblk0p1 partition on the chip, is the MSDOS FAT32 partition with all the boot loader code, kernel, etc.

Assessment

Well, it’s only been running about an hour now, so this is very preliminary.

First off, FireFox works (yay!) unlike in the last update to Devuan 1.0 or the too painful to use arm64 Devuan 2.0 install. Not super fast, but fast enough. I’m composing / posting this article using it.

It is a little slower than the full armhf system. Not much, just enough to notice. Likely as the 64 bit kernel is not as highly polished as the 32 bit and is moving 2 x as much data for every word fetch. Still, it’s better than nothing (the armhf failed install) or the fully fat and slow all 64 bit.

I’ve not seen any technical issues from running (yet…) and everything seems to run stable and well.

I’m going to be using this as my basic test drive workstation for Devuan 2.0 for a while. I’ll sporadically swap to the Devuan 1.0 Daily Driver for “compare and contrast”. Just to see if my speed perceptions are accurate, or being shifted by subconsciously comparing to the XU4 or Mac ;-) Basically, I’m going to keep myself calibrated as to what the experience is supposed to be.

This particular chip did NOT impress me as being the fastest when I was copying GB onto it. Also, having it chopped up into 7 file systems and putting ALL I/O load onto it is also likely less than ideal. When I last did a “file systems on real disk” test, some year+ ago, that made a BIG difference. So for the rest of tonight and on into next week, I’ve got a 1 TB USB disk I’m going to be formatting and setting up to take a load of the I/Os and see what happens to speed then.

In Conclusion

I’m quietly pleased with myself for making this hybrid 64 bit / 32 bit system. It isn’t all that hard, but it does take a certain attitude, and understanding, to think of “going there”. But that whole necessity thing was kind of in my grill. IF I wanted to test armhf Devuan 2.0, I either had to wait for someone to fix the released copy (I’m pretty sure now it wasn’t me causing it to not work), try it on a PC instead of an SBC, figure out what was wrong and fix it myself, or ‘get creative’. I chose creative as it’s more fun ;’)

This also lets me add all sorts of software without much worry as to what running on a back-level 1.0 release kernel would do (remember that I first got armhf Devuan 2.0 userland running on an older kernel here: https://chiefio.wordpress.com/2018/06/12/devuan-2-0-ascii-release-r-pi-install-tale-of-woe/) While satisfying that I could “make it go” and narrow the scope of where the problems were located, it limits the ability to reasonably test a new userland as any bug could be old kernel mismatch issues. Where 32 bit userland on 64 bit kernel is supposed to work.

Now you know why I’ve been somewhat quiet the last couple of days and not commenting quite so much on the political scene. I’ve been up to my eyeballs in bit shoveling ;-) Now that I’ve scratched this itch, I can come up for air and see what has happened in the world… other than World Cup and Trump Bashing… Or maybe that’s the empty set right now ;-)

Subscribe to feed