“Dead” Boards
Some folks might remember some years ago I got an Odroid C2 and loved it. Then it mysteriously “died”. Then a year or two later is was working again. Then the Rock64 “died” (it is still in a “Dead Boards” box). Then several months back the Odroid N2 “died”, and now I’ve figured out that while it LOOKED dead (not booting, nothing on screen) it was really hung in boot on /etc/fstab changes.
Well, the N2 was too expensive for me to just ignore, so after a certain amount of “set it aside for me to cool” time I whacked at it for a few days and figured it out. Now I’ve installed Armbian on it; that seems more resistant to that particular SystemD / fstab interaction bug. (I’m not willing quite yet to say it has no chance of it…)
So now I’m wondering if the mysterious “death and resurrection” of the C2 was a similar thing. This has encouraged me to dig the Rock64 out of the “Dead Boards” box and try a re-spin of that OS to test it, too. The particularly pernicious nature of that bug would mean that, for example, if you removed a disk (and it was noauto 00 in fstab so you think it need not be connected…) even if you restored a back up of the OS, it would still “act dead”. I did several restores of backups for each of those boards…
So sometime this month I’m going to put a day into recovery of the Rock64 with that in mind.
Why mention this?
This particular bug is strongest, near as I’ve been able to tell, in Ubuntu. BUT it could be in any SystemD operating system. Ubuntu (and other SystemD infested things like the “upstream” Debian) are THE most common OS shipped with the boards or provided on their support sites.
LOTS of folks are likely to have experienced this.
So IF you have a board “suddenly die”, it has a good chance of not really being dead.
I’m a natural “parts” packrat, so didn’t toss mine out. Other folks? If it is “dead” they toss. So don’t toss, OK?
When the C2 came back to life, I had no idea why. IIRC I’d just thought “Well, OK, lets just try a pristine OS copy”… which would argue for the fstab thing being causal as it would be gone then, but present in the backup copies (needing only a disk removed from the system but the mount description left in /etc/fstab and why would you plug back in a disk you had removed?…) I still have those old backup images, so I can test this thesis (but not any time soon… too much other to do).
I wonder how many hundreds of working Single Board Computers have hit the scrap heap from that one bug.
So, I would advise that as soon as you have the board booted up, change the configuration so that it does NOT give you the blank screen during boot, but gives you the running commentary and a login prompt for maintenance if needed. IF you already have a “Dead Board” try editing the uSD chip on another system to comment out any disk partition mounts.
Other OSs
I spent a good chunk of yesterday trying a Gentoo Linux installation on the Odroid XU4. Gentoo starts with a core “tarball” and you unpack it into a “chroot” partition where you can “chroot” into it and it looks like it is a new isolated computer. (chroot is a way cool thing for security. It lets you turn one set of hardware into what looks like two isolated computers. It was a key part of how we kept projects isolated on the Cray. Folks on the project side could not see the systems admin side, for example.) Then you do a build of the rest of the system from source tarballs.
I really really like the idea of building the system from source. However…
Their package manager, portage, is a PITA. It has too many knobs and configuration flags. I know, I know, it’s just enough for “the experienced operator” to get it to run on a a zillion different computers and just the way they want… But…
After about 6 hours of compiling things and completing something like 138 packages, it barfed on the next step. Some config line in one of the half dozen (or more?) configuration places isn’t right. Or something.
Near as I can tell there’s a missing directory / file:
>>> Unmerging (6 of 11) acct-group/render-0... >>> Unmerging (7 of 11) mail-mta/ssmtp-2.64-r3... >>> Unmerging (8 of 11) sys-devel/binutils-2.30-r2... * Switching to armv7a-unknown-linux-gnueabihf-2.32 ... [ ok ] !!! Section 'gentoo' in repos.conf has location attribute set to nonexistent directory: '/var/db/repos/gentoo' !!! Invalid Repository Location (not a dir): '/var/db/repos/gentoo' * Please remember to run: * # . /etc/profile >>> Unmerging (9 of 11) perl-core/File-Path-2.130.0... >>> Unmerging (10 of 11) net-mail/mailbase-1.1... >>> Unmerging (11 of 11) sys-devel/automake-1.15.1-r2... Packages installed: 206 Packages in world: 0 Packages in system: 43 Required packages: 206 Number removed: 11 * Regenerating GNU info directory index... * Processed 96 info files. odroidxu4 /usr #
These are the packages that would be merged, in order: Calculating dependencies... done! emerge: there are no ebuilds to satisfy "app-portage/gentoolkit". emerge: searching for similar names... emerge: Maybe you meant any of these: app-portage/gemato, app-portage/portage-utils, app-portage/elt-patches?
So where is repos.conf file? Unknown. They don’t give the full path name. One hopes it is the same as the config file that was edited per the “DIY write-up”. “The experienced operator will just know” is the usual sniditude applied to such circumstances. To what ought it be changed? Unknown. Ought it be changed? Unknown.
I’m hoping that I can just make the directory. That it doesn’t need to have anything in it, it can just be made and that’s it. Will that work? Unknown.
FWIW, I’m following the not-quite-complete cook-book here:
https://wiki.gentoo.org/wiki/Xu4
It has a different set of choices for a few items than those in the default as it is found in the tarball. I suspect this is where things go off the rails. I used the settings in the page to replace those in the tarball. This is done in a “repos.conf” file that one hopes is the same “repos.conf” file the error message is talking about. But maybe it isn’t… There’s a bunch of available config files and some of them seem to have the same or similar names, just different paths (per various pages I’ve read). Or maybe it’s the same file, just they “configured it” to a different location, or the location changed over time. (This is WHY I complain about capricious changes. It leads to confusion “downstream” or “later” unseen by the high dick-with-factor upstream developers [cough, Pottering, for example on the Debian side}).
There is a HUGE advantage to consistency and standards for things like the name space, location of files, methods of operation. The BSD folks “get that”, but are accused of being old fuddy duddies for it.
CHOST="armv7a-unknown-linux-gnueabihf" PORTDIR="/var/db/repos/gentoo" DISTDIR="/var/cache/distfiles" PKGDIR="/var/cache/binpkgs"
As unpacked, those values are not in /var.
https://wiki.gentoo.org/wiki//etc/portage/repos.conf
So can I just mkdir /var/db/repos /var/db/repos/gentoo and be “good to go”? Unknown. But I’ll give it a try when I’m interested in wasting a day or two, again…
Oh, and that day of compiling things isn’t all lost ( I think) in that I can just make a tarball of it and stuff it off on a disk as “XU4 base install plus make world” and pick it up again some other day.
I got Gentoo to work on the Raspberry Pi some long time ago, so it isn’t like I’m a complete Noob with it. But it really does illustrate the “User Hostile” side of Linux. I appreciate that their whole thing is “build it all from sources” so you know it is clean. It avoids all sorts of potential hidden Bad Things from Bad Guys. But really, would it hurt them all that much to have a pre-built image for the dozen or two major platforms where a Noob could just “download and go”? It would remove a huge barrier to adoption. Then, over time, those folks could expand into the build from source mainline… WHILE having a working model to study and learn from.
A similar thing afflicts Slackware. They pride themselves on NOT having a package manager. As though forcing folks to keep track of a zillion dependencies that are most readily found once and codified in code is “a good thing”, when it isn’t. Oh Well.
So, for now, for whatever reason, it looks like the XU4 is not a widely supported bit of hardware for “variety” OS choices. There’s some DIY Kits (like Gentoo & BSD) but for “unpack and go” it is mostly Debian, Ubuntu and Armbian – all SystemD afflicted, or the Devuan port that is’t working right. So also, for now, I’m just going to leave it Armbian and “move on”.
There’s a LOT more “variety OS” support for the RockPro64. So I’m going to move my quest for “Variety OS free of systemD” over to it. Their site at least claims a working Slackware and BSD image. We’ll see. Sometimes they are just pointers to DIY recipes.
Also, it looks like there’s a lot more OS chioces on the Odroid C2. Now it isn’t the Hot Hardware these days, but… it’s faster than the R.Pi M3. So on the Odroid front I’m moving my Variety OS quest onto it (where it is more likely to succeed).
Hopefully this will let me get something free of SystemD running on either the Odroid C2 or the RockPro64. Yes, I’ve got a wonderfully working just fine Devuan on the x86 PC and on the R.Pi M2, M3, etc. So that’s nice and they are all taken care of. I’ve also got a “workable enough” Armbian with systemD running on the Ordoid N2 and XU4 (and without that fsck / systemD interaction bug) so they are “nervously workable” and I’m just going to leave them that way for a while. (Using the XU4 to post this BTW). So that’s a pretty good collection of “things that are working well, seem stable and reliable” and don’t really need more futz with factor.
BUT: As Devuan on the XU4 was clearly at a minimum not really tested well at all and buggy: I want a choice other than Devuan for a SystemD free OS.
So I’ll be whacking on The Usual Suspects, but on the “other” platforms (so Gentoo, Slackware, BSD on RockPro64, Odroid C2) and we’ll see where it ends up.
“I love what Linux / Unix let’s me do, but I hate how it makes me do it. -E.M.Smith”
Musings from the Chiefio 
Interesting. The RockPro64 was not seeing disks nor uSD in carrier adapters plugged into the USB port. I swapped to a different installation (Armbian Buster / Sid) and it works.
This is one I was using back about May, and I thought it was fine, and it is. Yet the OTHER one, that isn’t working quite right, is also “Buster / Sid”…so something is not quite right in the details. Perhaps in the specific device tree / kernel modules available.
Whatever…
I’ve got it working on the install I’d had working before. It had (briefly) failed to boot, so I copied the image to disk and used the uSD card for other things. Now, I’ve put the image back on the same uSD card, and it’s working.
Next up, a test of it with the mount / partition lines in /etc/fstab NOT commented out ;-)
EM – is this helpful?
https://bbs.archlinux.org/viewtopic.php?id=192991
@Jim2:
It does look like that would fix the problem once you have booted up.
The problem is that it blocks booting up to even figure out what the problem is…
So I’m suspecting it is a bug in that it isn’t properly executing that command at boot time like it is supposed to do per the descriptions.
FWIW, tested it on the RockPro64 / Armbian and it is happy to boot, too. So looking ever more like an Ubuntu bug.
At this point my (very temporary) failure of the RockPro64 to boot this particular uSD image is looking ever more like a PIB-KAC error.
(Problem Is Between – Keyboard And Chair)…
I likely didn’t have the uSD card seated all the way or, I noticed in this test, my thumb tends to hit the reset / whatever buttons when holding the board to plug in a USB device and I may have toggled some state.
So, summary: Armbian is working FINE everywhere. RockPro64, Ubuntu N2 & XU4.
Ubuntu “has issues” with the /etc/fstab at least in that particular release, and it ends up looking like a “dead board” when it isn’t.
All of my “main boards” are working nicely, just not on the OS of choice for the fastest hottest boards (but one that is quite acceptable). So basically, I don’t have that failure to launch issue anymore and just need to make sure to test for it on new systems (and avoid Ubuntu… which I tended to do anyway but had exceptions.. like vendor installed.)
With that, I’m taking a break from this mornings hacking for a short news show cycle and a “beverage of my choice” ;-)
Then it will be trying NetBSD and / or Slackware on the RockPro64 since it is shown to be working fine (I’m using it right now to type this…)
It would be nice to get some alternative non-SystemD non-Devuan running on one of the Hot Boards, just as a proof of concept… and then for comparative performance and freedom from bugs. But that’s for after a short non-commercial break ;-)
Glad to hear is was a (relatively) minor issue.
A different minor tech item.
The Australian federal government document upload system seems to have been fiddled to where you now can’t upload via Firefox.
@Another Ian:
So Google have a big influence? ;-)
@ALL:
Some Very Good News!
I’m typing this from Slackware on the RockPro64 in a GUI environment!
It is just sooo wonderfully familiar. Things where they ought to be and all that.
At first boot up, the power light came on. Then the CPU starting light came on. Then a minute or so later it all went off, then came on again… then “nothing happened” for a long time. I went off to do something else and what I think happened is my TV set went to sleep.
Too long with no signal, it shuts off.
I thought the install had failed. From another terminal I checked the router, and it said that system had an IP address and was running. I power fail / rebooted… THEN discovered the TV was off. Powering it on and rebooting, It came up No Problem.
The default user is root, so that’s how I logged in. Tried to run FireFox. NO Joy.
Damn, browser doesn’t work, I think. Then decided to launch it from a text window instead of the menu dropdown. bash-5.0# /usr/lib/firefox …. and it gave me a “Don’t run browsers as root!” nag and stopped. OK, not a bug, feature.
So I did “adduser” to add a regular user account, and then firefox was fine.
The system comes up with a clean xfce GUI. I’m OK with that, but prefer lxde. Maybe I’ll change that later…
There was one moment when I started typing this that firefox took one core to 100% and was unresponsive for about a minute. I think it was fondling it’s updates stuff. Since then it has been fine.
I did have to set the system clock to something sane…
Oh, and video compositing is lousy. Like they are not using the GPUs (same issue as some other newer boards). Dragging a window sort of shudders across the screen, lagging the cursor. A minor annoyance that will be fixed some day.
So, in general, I’m VERY pleased. With a fairly painless install, I’m running a nice old Unix v7 (BSD) style rc.d init system with no SystemD anywhere in sight! Gotta love it!. Oh, and sound works on youtube videos too ;-)
I’m going to keep one Armbian uSD card for the RockPro64 just in case there is something I need that this release doesn’t do, plus I need to learn how to add programs (as Slackware is not fond of package mangers with automatic dependency management…).
BUT, this is going to be my default first OS to try on the RockPro64 for any given tasks.
So far it seems generally fast and efficient (modulo the video compositing). But I don’t drag around big windows all that much. I tend to set up the tiles and then just leave it.
So count me a Happy Camper ;-)
Oh, and it’s running a full screen video at 1080p ;-) with a load of CPU left over ;-)
E.M
Pretty sure it will have Microsoft , not sure about Google
Slackware solved my, until you mentioned it, load problem. I may also have several bad uSDs.
@corev:
Slackware does seem efficient and clean.
I’m keeping 2 uSD systems in play for the RockPro64.
Armbian – because it mostly works (sound is gargely) and I know how to add programs.
Slackware – because it is SystemD Free, has GOOD video, and seems solid.
I need to redo my BSD test as it, too, seemed to boot but not have video… but with Slackware I changed the TV to look at a TV input and when I changed back, the Slackware screen was visible. So I think there may be a bad interaction of the TV Timeout timer and the first boot actions. IF I can get a nice BSD running too, that would be a third.
As soon as I’m up to speed on adding packages to Slackware, it will be the #1 on the RockPro64.
I intend to carry this pattern forward on any other board that doesn’t have a Devuan port. Devuan stays #1 on the PC and R.Pis. Other stuff will get an Armbian Desktop (as it seems to work on everything) while I work down the list of Slackware, BSD, Gentoo looking for what comes up clean and easiest.
While I’m skilled at Admin on everything from BSD to Version 7, to System V, to Linux of many flavors: I’d like to keep the zoo as small as possible. They all have quirks to track… (At least I don’t have to deal with Sun OS vs Solaris vs HPUX vs Irix vs Ultrix anymore :-)
The Armbian Desktop is an Ubuntu, so not on my preferred list, but the Armbian guys seem to have caught and squashed most of the bugs and port it widely with decent performance tuning. So it is a nice quick “bring up the board” system while the harder to install ports are made right…
Per uSD cards:
I’ve had 3 SD to USB adapters fail (one going flaky first) so it might be the adapter more than the uSD… Just sayin’… test it in a camera or some such or try a different adapter when flashing the OS.
Well this is a minor bother….
I flashed a new NetBSD image to uSD, and booted the RockPro64. It does the wait / blink lights / wait cycle and ends up with green power and white running LEDs.
The Router reports it has an IP Address.
Doing an SSH to it connects and asks for a password. But…
The directions say root has not password. That doesn’t work. Nor do any of “the usual” default root passwords. Further poking shows that by default, SSH is blocked… So, OK, where do I log in? Most likely on the “Serial Console” that I don’t have.
Linux does understand the ufs file system, but ONLY for reading, not writing, so I can’t even mount the uSD on a Linux box and change the config that way.
So, how convenient, they ship an image that does not use the major terminal port (HDMI) and won’t talk to you via SSH / remote login. Nice locked box you got there…
So no, I’m not breaking out the soldering iron to kludge up some kind of serial console connector, and no, I’m not going to try some more friendly BSD on the PC to then leverage it to edit the config on the uSD to either turn on HDMI or enable SSH. Nope.
What I’m going to do is just Walk On By…
I really do like BSD, a lot, but the folks packaging this stuff need to get it through their thick heads that the reason they are heading for the dustbin and Linux is taking over the world is simple easy user friendly installation. Not a land mine of barriers to entry.
So with that, moving on…
Enabling ssh for installation without any console. (There are other I/O options on the linked page)
If you want to enable ssh with the standard image, so that you can log in over the net without either a serial or HDMI console, you can edit the configuration of a uSD card before booting. On another computer, mount the ffs partition, place /root/.ssh/authorized_keys, uncomment PermitRootLogin in /etc/ssh/sshd_config, and comment out the rc_configure=NO in /etc/rc.conf. Besides having to find the IP address (e.g. from DHCP server logs), you will have to wait for the partition resizing and reboot.
https://wiki.netbsd.org/ports/evbarm/raspberry_pi/#index9h2
!@Jim2:
Thanks for trying, but that is Pi specific. It has a fat32 boot partition. The RockPro64 has, IIRC, only a hidden magic boot and a ufs partition. Nothing to edit on a non-unix system.
I’ll double check it, now that it has booted once, but I’d looked at it before with gparted and it was devoid of writable stuff.
Well I’ll be. gparted now shows a Fat16 partition.
The boot command has console=fb though, which I think means it is trying to use the hdmi already. So some ore digging needed. BUT, it looks like you are correct that I have some place where I can change boot arguments without a Unix already running. At least to some extent.
And mine has console=fb so ought to be going to HDMI. I think.
OK, this is at least a little strange.
The fat16 partition with the boot commands and such had a Broadcom license text and some broadcom files only seen on the R.Pi boots. So I stuck it in the Pi M3 and it booted fine with the HDMI output working (text mode). The “startx” command launched a 3 terminal window standard old school X session…
So I have a NetBSD that works on the R.Pi from a download for the RockPro64… that did boot on the RockPro64 as it gave me login prompts on SSH, just refused to honor them.
OK….
I suspect that BSD being more “generic hardware friendly” was running just fine (likely as v7 instruction set) across many ARM chips, so they just tried what they had, it booted, so they put it in a metaphorical box and shipped it…
I can test this later (MUCH later) by just putting a non-root user on the system, making sure SSH is working, and rebooting it in the RockPro64. But not, I think, today…
It’s also possible this is just a quirk and someone put the wrong image in the download area / link.
Whatever…
Mostly I’m just going to leave this (old 4 GB uSD) system “as is” and whenever I next feel like fooling around with a BSD, play with it on the Pi M3…
FWIW, looking over the general ARM build instructions, a lot of them look to build fairly generic v6, v7, or v8 builds to be used on several platforms; so the idea of one built for the Pi working somewhere else is not that odd.
My key takeaway, though, is that the advertized NetBSD RockPro64 image as downloaded didn’t cut it for what I wanted to do. So that’s a black hole of time suckage for the foreseeable future. Which means I’m “moving on”….
Summary:
I’m using Devuan wherever it works, so right now this is from the PiM3 on Devuan… For the Pi stack, I mostly just need to go prune / archive a lot of old cruft releases I’ll never run again and simplify the whole stack of archived “junk”. Housekeeping & archiving only; for Pi and PC releases other than the current Devuan 2.0 build that is just fine.
I (accidentally it seems) have a booting NetBSD for the Pi M3. Any desire to dink with NetBSD will be satisfied there; but not soon.
RockPro64 working on Armbian / Ubuntu well enough, and Slackware nicely. Need to put time in to Slackware admin skills rather than chasing install oddities in other OS spins.
N2 and XU4 both have nicely working Armbian / Ubuntu desktops. It is an open issue what I’ll get put in as the non-SystemD alternative. Need to check out Slackware, Gentoo,and BSD options, if any, on them. Not a priority at all, so likely next week, or later. At this point the time budget has run out on “other releases” efforts.
Did You Know, when someone at my level of ‘puter fu sees someone at your level saying something like “I need to learn how to add programs”, it appears in large, ornate, flashing red capital letters … ;-)
Seriously, that’s one thing MS (and the Debian package family, IME) have had right for ages – box comes up saying “Do You Want To Install DogSoft v3?”, with Install and Cancel buttons, and off it goes. Of course the end user has the right to expect the system to sort out dependencies and the like, and there’s no reason not to have decent security checks as needed behind the Install button. “I want to install X” implies “… and I want it to work”, not “… and I look forward to a fun few days trying to fathom out why it won’t work”, especially in end user land.
Re those boot-time weirdnesses, I sometimes think Uncle Sir Clive Sinclair and his pals had it well worked out forty years ago: if all else fails, boot from a ROM Basic (or ROM “something”) which will at least let you get in there and work out what failed. Ah, progress!
SteveC, I agree: ” I need to learn how to add programs”, it appears in large, ornate, flashing red capital letters … ;-)” Messed with Slack’s version for ~ 2 hours and then went looking for anything else to use. It’s just too different to spend time learning their unique functions.
Some Linux-based packages need dozens of libraries. I’m not sure I want to go through that just to install a program.
MS assigns each library a Globally Unique Identifier (GUID). This way multiple versions of one library can be installed side by side without stepping on each other. I agree this is something MS has done right … very right. I’ve read there are some programs that make installing to Slackware easier, but don’t know how well they work or if they are well supported.
@Steve C:
Yeah. Not fond of the Slackware Way on dependencies. But it is HIS release so it gets done HIS way. The assertion is that you can have failures if the automation is wrong and then it is a nightmare. I think “Yes but, rare…” ( I have had it once long ago…). So I suspect some early package manager bit him and now…
FreeBSD has it done. Period. Gentoo has it done, with so much glitter and options that you will spend months figuring out how to operate it. Debian family (so includes Armbian, Ubuntu, etc.) has it mostly right (except when the Devs push a broken release and then you hang your system after an update / upgrade and get to figure out how to recover…)
But “It’s good for me” ;-)
Per the boot: Well, in fact, the default is to do just that. There’s a configure option to hide it. THAT’s the mistake, hiding it. Normally it boots the initramfs and then goes off to look for things like disks and such. IF that fails, it gives you a prompt to log in as root and do maintenance or Ctl-D to continue anyway (so if the “fatal” error shown upscreen was “could not find disk you removed yesterday and didn’t update fstab” you can just say “never mind…” and boot)
I REALLY wish the folks who try to make the boot process look like NOTHING is happening would learn that is a bad error.
@Corev:
Oh, so I have something to look forward To! ;-)
Yeah, not thrilled at the prospects. BUT, unfortunately, it is only a few Surly Curmudgeon releases that have defied the bogosity of SystemD, so you get to deal with their quirks instead.
I’d rather Devuan (old Debian before the SystemD disaster). BUT until enough folks port and debug it, the older more distributed releases are the only choices.
That largely ATM is Gentoo (4 to 8 hours to compile and a Great Swiss Army Knife Package Manager with nobs and dials and switches for EVERYTHING!!! GUSH!!! – /sarc;)
Slackware “we don’t need nt steeenking dependency tree!” with DIY bits.
I’m going to plow through both of them until something shakes loose (that might end up being my loathing of SystemD.
@Speaking of which:
Well, another day another SystemD Dumbness….
So I was adjusting the swap priority of some swap partitions to assure that when it swapped, the swap went to the least busy disk / partition first. Normal everyday thing a systems admin does over morning coffee.
I change the /etc/fstab entry, swapoff, swapon -a, swapon -s
Nothing changed.
I try a couple of more times as it IS a First Coffee moment. Nothing changes.
It seems that, likely at start up, SystemD reads /etc/fstab and from that moment forward the values are SET, PERIOD. No interactive changing for you…
There’s probably some obscure SystemD-ism (Dumb-ism?) to get around this. I don’t care, I’m not going to bother. This particular SystemD afflicted system is just an interim step to something else anyway, and tuning swap performance is thin gruel on this one as it never runs much over a 100 MB on ONE 2+ GB partition (the other 3 GB never being used). But still. Just a WT? moment….
(The genesis of this is that on the UX4 under this release, Armbian, it has a zram swap with priority 5. I had added Real Disk but at a priority 128 or so. I thought maybe since it has 2 GB Real Memory and 8 cores that are almost always idle, using compressed memory as the first layer of swap might be nice… so was going to move the disk to priority 4, just behind the default zram and test it by launching the browser that rolls excess pages to swap. But NOOoooo.. SystemD has other ideas…)
I’m running a backup job in another window or I’d just reboot and then test it… so I guess getting that answer will have to wait an hour or two…
In Other Excitement:
The reason I’m doing the backup is because the R.Pi M3 would not boot and gave library missing and other horrible errors. It boots with key directories, like libs /usr/lib /var on hard disk partition on my oldest and smallest (500 MB) disk. It was not seeing the disk.
Well, a few frantic trials later and then moving it to the direct USB port on the XU4 (that has the power to drive a hard disk without a powered hub) the disk is Just Fine. It’s the cheapo cheapo $25 powered hub I bought several years ago (and plug / unplug a few devices daily) that’s lost the plot on a few of the USB ports. (The keyboard and mouse rarely were plugged and are still working as I type this on it…)
So, for now, I’ll be moving the hard disk to another, lesser plug worn, USB port… but while I had the XU4 booted, I’m grabbing current copies of the 2 partitions that matter most. (Home Directory and Working Storage where I sling most bits most of the time).
This, combined, says I’m likely to deprecate using the hard disk on the Pi M3. First off, I don’t really need to. It was just to reduce uSD card “wear” and I’m now using other boards much more often, so likely best to just “move along” and go back to the partitions on the uSD card. Also, this particular uSD was when I was first playing with the 64 bit build. I have both a 32 bit HF userland and a 64 bit userland over an Arm64 kernel image on it. It’s been a year+ since I bothered to boot the 32 bit image. Arm64 is working nicely, now, on the Pi. So really I need to just “move along” from this hybrid chip anyway.
So, unexpectedly, I’ve been nudged to “clean up the legacy” on the PiM3 and simplify it. While it was great fun having an Arm64 kernel and being able to boot both ways, both userlands, and while it was useful then to have high wear partitions on the old hard disk; now it’s a “risk factor” to booting (until I get a new USB hub…). And for what? Not much, really, now.
I’m a lot more likely to be using the XU4 or one of the other “Hot Boards” for anything with a “load” (and that includes browsers. The Pi is still very important for those times when it has stable bug free software that hasn’t made it to the newer boards yet (like Python graphics libraries) but that can be on a different disk – I have some with external power bricks that don’t need a powered hub).
Moral Of Story:
If you think your disk might have died, try it on another system, and suspect that $25 cheapo hub more than the $100 disk… (hey, it was new and expensive once ;-)
Interesting rule of thumb….
Just ran a ‘tar cvzf’ backup on 70 GB and it took a long while. about 2 hours (disk to disk USB 3.0). Just to see, I ran the same thing NOT compressed and not putting anything on the screen “tar cf”. The product was about 2 x as big, but took about 1/2 the time…. This was on the XU4 where the compression tended to peg one core at 100% and almost always one of the BIG cores (A15 IIRC).
So first off, this implies it would go faster n the N2 or RockPro64 (A73 and A72 64 bit cores) for the compression where one of those cores is a lot faster than any other cores I’ve got (A15, A56, etc.). And secondly, it implies that as a general fudge factor figuring your compression costs about the same ratio of time is likely a good first approximation until you can benchmark a particular bit of kit.
So: BIG or fast; pick one.
Interesting….
When shopping while the Backups ran. I’d done a ‘swapoff’ for all but the zram partition. Now, a few hours later. Edited /etc/fstab to assure real disk priorities were set low. Did “swapon -a” and then ‘swapon -s” to see the result.
So now it’s clearly read the /etc/fstab and reacted accordingly. So SystemD has some kind of timer going on as to how soon it will re-read /etc/fstab changes. Well that’s a mistake…
Whatever….
In any case I now have the ladder of swap priorities as desired. Use zram for the first GB (and likely the only one…). There’s 2 GB of memory so I’d guess this could reduce available memory to about 1.5 GB as the zram grew (assuming about 50% compression). Then it rolls to the 2/3 of the way down the disk 2 GB partition for the next 2 GB. This is the spot where head seeks are most minimized.
At that point, you have 3 GB on swap and about 1.5 GB of real memory. That’s about the limit for normal / good performance. BUT, just to prevent crashing, there IS another 2 GB of swap near the front of the disk available…
IF things are ever running over 2.x:1 swap to real used memory, you are in a world of hurt usually disk thrashing on swap. But at least you are not crashed.
This lets the Systems Admin (yes, slowly, painfully, patience as each typed command may take 2 minutes to swap in and execute) start killing off processes and fixing whatever is in runaway memory usage…
Originally I’d put zram at the end, figuring that 2 GB real available memory was better than 1.5 GB. But after a lot of use, it rarely to never gets over about 300 MB on swap. So about 150 MB of Real Memory ™ taken for swap in exchange for Damn Fast Swap based on excess computes (the compression / decompression.) I’m thinking that’s a good deal…
In any case, that’s the config on the XU4 right now. We’ll see how it works in practice.
Sidebar On Preferences:
The specs on the RockPro64 are spectacular and the two A72 cores are damn fast. It’s just a wonderful bit of kit. The N2 with 4 of the A73 (slightly faster than A72 and a LOT cooler so they can put 4 on a die) is just smoking. Both are 64 Bit. Both have some (A53 like the Pi M3 IIRC) “little” cores for light demands. They are in every way superior in specs to the XU4. It is “just” a 32 bit…
BUT, I find I just like it more.
Software is more mature. Then 32 bit is just fine for 90%+ of most of what anyone does. It’s small. It’s fast. It just “goes”.
Then, with 8 cores, the theoretical max computes is not that far off what the other two boards can do. It does multitask well. Having 4 big fast cores and 4 little cores (where little isn’t that little) it handles a lot of things well and really like multi-thread codes.
Part of it may have been the “Armbian uplift to Devuan” that I was running on it. It was the XU4 or the R.Pi to avoid SystemD. We’ll see if the admiration continues now that I’ve moved to straight Armbian Buster…
But whatever. I find I just spend way more time running the XU4 than running any of the other boards. Faster than the Pi M3 (while being as robust to software quality generally). More bug free than the newer N2 and RockPro64 (perhaps due to Ubuntu “having issues” with /etc/fstab on them tainting my experience… we’ll see now that I’ve moved them to Armbian who have cleaned up that bug.)
There’s something to be said for “it just works more of the time and with less pain”…
I find myself wanting to buy another one (“as a spare”…) even though I don’t really have a use for 2 of them (yet). My explorations of cluster computing have yet to show a big win case. One Big Core still beats the stack. Software issues? Implementation? Wrong choice of codes? Who knows… So it isn’t like a stack of 8 of these is calling my name…. (but it would only be about $480 … and have 64 cores…)
BUT: Until I have a clearly defined and proven “use case” that the stack / cluster solves, it is just self gratifying indulgence. And I usually try to avoid that.
Were I putting down my money, right now, on Just One new SBC? It would likely be the Odroid XU4Q Quiet model without a fan but with a huge heat sink. Runner up, the RockPro64 (as it has a lot of OS choices…). That may change 6 months out as more OS choices show up for the Odroid N2. It IS superior hardware. BUT the software choices are limited. Ubuntu cripples it, IMHO. (As evidenced by mine sitting in the “How the hell do I make this run again?” box for several months). I’m hoping that Armbian tames it… I can see it as the eventual “hot babe” that pulls me away from the “first love” of the XU4.
But not, I think, today…
Another Day another weirdness that is likely in some way related to SystemD and the changes for it.
So I log onto the RockPro64
Armbian – straight with SystemD:
and it has a disk plugged into it that isn’t in /etc/fstab. This shows up as an automount thing on the desktop on a lot of releases. Not on this one. I proceed to do my usual launch of htop (to monitor activity and processes on the system that helps with both performance tuning and with intrusion detection as “hammer on it” attacks show up as unexpected load) along with a root terminal.
In the root terminal, df shows the disk automounted. In another, non-root, terminal, it doesn’t.
So exactly when did the folks designing this decide that lying about system state was a good idea?
So I was going to look at this listing as I typed stuff into /etc/fstab to make sure I got them all, but when I popped open the non-root terminal, they were gone. Then I did an SU in that window and, bingo, there they are. GAK!
The ‘df’ command is supposed to show me what disks exist and how much stuff is on them, not lie to me… Had I just thought “Oh, the disk isn’t mounted” and pulled the USB plug out, God Only Knows what would happen to disk status. An UN-mount is an important process to NOT skip…
Every day
In every way
I’m hating SystemD more and more…
AND the people who designed this mess.
E.M.
“Every day
In every way
I’m hating SystemD more and more… ”
Sounds to me like “amoeboid software – poke it here and it gets you somewhere unknown”
@Another Ian:
The basic problem is that SystemD is NOT an “init system”. It is a Service Manager that thinks the whole Operating System is a “service”. So it layers on an alien systems admin system next to, under, and over the traditional one. Now you get a squared function on “issues” and “where to do something”. Just try to find, now, how to get the OS to NOT boot up to an automatic login. You get about 1/2 dozen answers. All of them right some of the time and all of them wrong some of the time depending on a bunch of variable state and history of your system. In the past, it was just change lightdm.conf and move on…
You get the product / exponent of all bugs, misunderstandings, errors of configuration, changes in updates breaking things, etc. etc. So in this case, in some releases, the system hangs with a black screen in boot on “noauto” entries in /etc/fstab. On others, they fixed it. SystemD Developers really think you ought to use Their Way so ignore (largely) making fstab handling right, further expanding buggy behaviours.
It is just a dogs breakfast of a mess.
At this point I’m leaning toward giving Gentoo a go. Slackware “had issues” on the newer SBC (as do most OSs since the new boards have new ports with the lowest bugs patched post port…) but likely works FINE on PC / x86 boxes. Slackware never has really embraced the ARM world. For a very long time it was a “community port” and still is a second class citizen there. In the Gentoo world they embrace multi-arch at a core level. A bit messier to get it installed, but maybe that’s a “feature” and once I’ve worked it out I can publish a “Spin” or “Distro” of it. ;-)
Devuan is great, on the PC and the Raspberry Pi boards. But they are resource limited and can’t cover many “variety” boards (nor, it seems, do full QA on the Odroid XU4…). So, as of now, I’m using Devuan wherever I can, and still deciding what’s to be used on the stranger boards. For now I’m gritting my teeth and using Armbian as they seem to have hidden more of the SystemD Crap than Ubuntu. (They fixed the fstab thing and the renaming of eth0 is set to OFF keeping traditional names by default). So since Armbian seems to have it working and right, I’ll use it until I get a working Devuan or alternative (such as Gentoo without too much pain…)
This blog has a longer list of choices than the 78 listed by Distrowatch:
https://sysdfree.wordpress.com/
and I expect the number to increase over time.
Those who would convert to SystemD have and the rest are finding out it was the right idea not to Go There. Those who had it forced on them are regularly reminded why it was a bad idea.
That usually goes one of two ways.
a) The new thing gets all the problems fixed, is better, and dominates.
b) Folks slowly abandon the buggy thing over time.
It is my assertion that SystemD is so fundamentally flawed it is not possible to fully fix it. So B ought to win over a few years.
It is also possible that the market will bifurcate. Red Hat and other Big Corporate Providers stick with SystemD as they actually want to have rapid spin up of VMs and have a staff of hundreds to deal with the crappy bits. Then the Small Biz / Home Systems who want something simpler, reliable, and low maintenance load.
But we’ll see. The choices on Intel PC hardware will generally decide it all, and they have LOTS of stable working non-SystemD options.
I’ve chosen to live on ARM hardware, generally. That immediately puts me in an Edge Case as most Linux distributions support ONLY Intel x86 / AMD64 hardware (and increasingly not x86 and only the 64 bit Intel AMD options). So “folks like me” on ARM boards will not determine the market future.
Then, on THE most popular / sold ARM boards, you get a large enough community to make it work well. So the Raspberry Pi is flooded with options (and Devuan is FINE on it). But I’m using some of the “Latest & Greatest” new boards with new CPUs and few sold. That pretty much guarantees one or two distributions ported to it so you take whatever exists, and a load of bugs as nobody has used it long enough to find them, never mind fix them too.
The point being that my experiences on the Odroid N2 XU4 and RockPro64 don’t mean much at all for the general market NOR for any particular Linux distributions. It almost entirely reflects how new the boards are and how many are sold. Which is why I’m likely going to end up doing a Gentoo port. Gentoo is frequent on small boards, often an early port as it is by default a source code self compiled port, and is widely ported to most of the quirks of systems have a way to handle it already in the source code somewhere that just needs activating / selecting (but comes with the obligation to set all those selectors / switches…)
Oh Well. It was my choice, so is my choice…
I’d hoped / expected that Devuan would show up on more total boards faster. It hasn’t, so I’ve got to explore some options (that could include me just doing the Devuan Systems Programmer work of doing the port to my own hardware… but I’d rather not. I’m only OK at that level of work.)
In theory, it’s ready to go on 9 or so of my computers. PC, R.Pis, Orange Pi, and I think Odroid C1 as well. It’s only the new fancy boards and the Odroid C2 that I think are not supported at all, and the XU4 that is supposed to work but “has issues” (i.e. bugs to patch). I still think that in a year or two more Devuan will be even more supported (fewer bugs, more boards); but until then I’ll look for some other option on those boards. Be it Armbian-gritted-teeth-systemd, or Gentoo, or whatever. Armbian is OK and livable, so it isn’t like I’m highly motivated on any given day ;-)
I’d thought Ubuntu was in the same category as Armbian, but their last two major Aw Shits have sown otherwise. (Failed blind boot, renaming all your interfaces to bizarre stuff). So Ubuntu is off the list for me.
Well, that’s a bit much for this. Time for me to get back to work…
A minor observation:
I was using the Compaq Evo this morning (to try to recover a uSD card that’s 32 GB but stubbornly insists it is only 900 MB now, even though Windows and my Camera see it as 32 GB, Linux sees it differently, even after a format. Ideas welcome…)
Now, rebooted in Linux, I’m watching a video (the SAFIRE one). and noticed an odd thing. It is a Pentium 4 single core. The video is running at 360p wiith some jumps and jitter. The SAME THING I get on some of my smaller SBCs that causes me to think they are “way too small” for things like video or big computes.
So what this is telling me is that the $15 to $20 SBCs I’ve got are basically caught up with a PC of not that long ago. In fact, were it not for the fact that the monitor I have only works with the serial interface on this PC I’d likely never use it. (Even the Windows XP on it is losing use as a “If I need Windows for something” thing.)
This also says that the price point for “more than enough” computes for things like full motion video at 1080p is about $60. Maybe less. It DOES depend a lot on the question of use of the GPU by the operating system and the application. Chromium on the Odroid N2 loads up most of the cores for video while FireFox only uses the 2 small cores – clearly FireFox knows how to use the GPU. On the N2, both work fine, but with very different CPU loads.
Sidebar On SystemD Follies:
I’ve made a square grid of SBC Board vs OS Choices and started filling it in with what I’m actually running, and what I’d like to run to purge systemD from my life. It’s a bit dismal.
What has become clear is that most of the board makers ship OS types with SystemD (usually Debian or Ubuntu) especially for the less widely sold boards, then Armbian works on just about everything but is also SystemD. Since the SystemD folks (Pottering – blech…) have decided to extend their screwage ever wider, release by release, it’s basically not tenable for me to continue to use Armbian as the “patch” to keep a board in play AND up to speed with software releases. So it’s a 4 way choice:
1) Install a non-SystemD linux.
2) Become “out of support” (and need to grab my own package repository image / mirror).
3) Stop using that SBC. (Not a big cost for things like the Orange Pi & Pine64 at $16 or so)
4) Roll My Own distribution (Linux From Scratch or based on a non-SystemD “upstream”)
I’ll be doing a mix of those things.
Clearly #4 is a huge time sink and I’m already way over-committed and short on “spare” time, so it’s a “someday thing”, maybe. I’ve done a LFS install on the R.Pi, and I’ve built Gentoo (a source build without systemD) a couple of times, so it is a “possible”, but I’d rather it were a last resort. I could easily see a compromise where I use the kernal and Device Tree (DTB) blobs from, say Armbian, then just layer on a UserLand from some other source ( I’ve done that with making a hybrid 64 bit kernel / 32 bit userland on Raspberry Pi / Devuan). So in any case, this is a Slow Boat Operation.
#3 is pretty easy and to some extent I’m forced to do it just via desk space. At any one time, I’ve got 3 or 4 SBCs in a box doing nothing. Easy enough to focus that on the lower cost, less widely used, systemD OS Only group. So, for a little while, the Pine64 A64 board (about $17) and the Orange Pi One boards (2 of them) at about $12 each will be headed for a box. IF Someday I’ve got a solution for them, they can come back out. Also, should I ever need something like a headless NFS server, I’ll toss BSD on one of them. (NetBSD is just about everywhere ;-)
For #2, that’s an option for some uses for about 5 years max. Things that are not Internet Facing can be left without the latest security patches and for many use cases you don’t need to worry about things like compatible libraries (but you DO need them to match in things like distcc distributed compile clusters). It is likely that the Odroid C1 and perhaps the C2 could end up in that state. There’s not a lot of OS choices for the C1, and as it is older, unlikely there will be a lot of new activity. The C2 is a bit better positioned. Still, i’ts a certain level of work to keep them up to date and there are not a lot of systemD free choices for them. Basically, it’s a low priority to keep them running and current.
Which brings us to the #1 choice of using mostly those SBCs / computers that have a non-SystemD OS available and that works well. My first preference is Devuan. It has nicely running releases on the PC (that’s now uninterestingly slow…) and the Raspberry Pi boards I have (5 of them). So that’s a reasonable base case. It claims to have a 2.0 release for the XU4 but I’ve not gotten it to work yet. So debugging that will be a priority. I’d be quite happy with that set as my predominant use batch and I’m pretty sure that, worst case, I just need to use tar and move the 2.0 XU4 userland to a known booting kernel / DTB / uSD card.
Which leaves the Problem Cases. These are mostly the newest and fastest SBCs where the vendor ships Ubuntu, Armbian has support, and not much else is on them yet as they are relatively new. (There’s also a couple of mid-tier boards like the Rock64 where I’ve not decided if it lives in a box unused or becomes a devo / porting test case). That’s the Odroid N2 and the RockPro64. Both presently running on Armbian with systemD. I can just leave them that way for 6 months to a year and see what develops in the way of other support, or I can “roll my own”.
Which puts us at “What are the choices?”
IMHO, and after a fair amount of digging at it, the non-Devuan choices are pretty slim. In order of my interest, they are:
1) Slackware. IF it works on a given board, it’s generally a reliable choice. Still uses the original BSD Style init (rc.d) and is generally “feature complete”. Support for ARM is low, but growing. It’s a bit of an annoyance how dependency is handled, but they do have some package management. It’s an OK option. But not one I’d want to port on my own.
2) Void Linux. An “up and comer” that’s slick, but has the teething problems of all things new. They are exploring the “different libraries” and other options more than broad porting. So if they have a port, worth it, if not… They claim an Odroid u2 / u3 port, and the XU4 is supposed to run U3 codes too, so maybe… A quick test in order “someday”.
3) BSD. It’s on most of the SBCs in some form. Biggest issue is just the very very CLI Old School all manual way of doing things like getting X-windows running and a GUI. A PITA, but works. I’ll likely start using this for various Headless Server uses (like my NFS file server on an Orange Pi) and eventually grit my teeth and deal with the GUI “issues”.
4) Gentoo. I spent a good day doing a Gentoo build / install on one SBC to then find that it was broken. OK… So I can become a Gentoo Developer (as anyone using it is, by default, doing a full compile from scratch and set all the knobs and fixing things yourself… I.E. doing “development”…) or not use it. Sigh. Someday it would be nice if Someone would make a pre-built debugged and GUI complete Gentoo distribution for folks who don’t want to spend a day (or a week…) getting it running on a $50 computer…
5) LFS. The Linux From Scratch process is rather like the Gentoo one, but without all the peculiar and release specific workload / detail / complexity of Portage. I’ve done it once on the R.Pi M2 and it was “OK”. BUT you are basically a Developer Of One…
6) Puppy? IIRC, Puppy Linux is not going to SystemD. Maybe. It’s a small light weight distribution that’s OK (though I find the naming conventions a bit treacley and their “look” a bit old school kiddie land). but it DOES work, and relatively well. I suppose I’ll just have to get over it and put a Puppy on some board and try again. Has a menu driven admin “feature” that I find annoying as I know how to do it all long-hand CLI, but nice for folks who don’t.
7) Armbian. As a very last resort, the Armbian folks seem to have fixed or shut off more of the bogus stuff than other distros (Ubuntu…). So I can likely just keep it going a year or two on the various boards where it is now. THE big downside is they have about a 6 month release lifetime. Then you are SOL. It’s upgrade or die. On the XU4, I just had Jessie leave support. Suddenly and abruptly can’t add new packages. I suppose that “somewhere” they have the apt sources.list targets in archives, but you either make your own mirror for very long life, or load up all the software you ever want in the first 6 months… As 90%+ of everything I do is met with some standard Linux Services, a build tool chain, and the Browse / Office / Gimp set, I’m likely OK on some SBCs with just “the usual build” and move on. Still, be advised. So this is my last option, but the one I’m using most at the moment…
So I’ve spent a couple of weeks fighting all that. It would be a LOT simpler had I focused on Just One or Two SBCs and especially the most widely sold / supported ones. BUT a big part of my goal was to find out what SBCs would work for a small cluster computer and what were the issues. You can’t find out the “issues” without having some issues… So I’m not bummed about it; it was a goal state.
That said, I have those answers now. So “going forward” for anything other than “exploring the issue space” I’ll look at the Devuan Supported list prior to adding any hardware. Clearly the easiest path would be using Intel Arch boards / boxes. For some personal reasons (like quiet and cheap) I’m going with ARM instead, then there are those security issues in the Intel… Then the Raspberry Pi is sold in such huge numbers that, despite their heat management issues, they are the #1 option. (Yes, eventually I’ll get a Pi M4 and a bunch of heat sink gear…) I’m hopeful that the Odroid folks get out of their Ubuntu / Armbian fixation and start some non-SystemD OS support. I really like their hardware ;-)
I am highly unlikely, at this point, to buy any new SBC that does not have a non-SystemD OS available. I just don’t need it, and I certainly will not go any further down the path of using a SystemD release as a last resort. The latest fiasco with Ubuntu and fighting both the fstab-makes-your-system-look-dead and the all-your-device-names-are-wrong headaches has shown it is just isn’t worth the trouble it brings. I’ll keep using my present Armbian installs for at most a year, then I’ll need An Option.
So that’s the Big Question: Which PITA DIY Option to do? LFS, Gentoo, or Slackware porting? (Or, very remotely, Devuan or Void porting / devo. Devuan has a strange porting system using zsh and Void is interesting but all different…and unlikely to port as widely as Gentoo). It does look like I’ll need to embrace one of them for the “variety boards” and I really only want to learn one strange package / build system…
I’m presently leaning toward Gentoo. While Portage is a complicated “everything and the kitchen sink” system AND building all from sources is a slow process, I do have the compute power to support it and it DOES work just about everywhere (once you screw around with it enough). LFS is similarly slow to compile AND you get to deal with every bug and porting issue all by your lonesome… Then Slackware is a reasonably good option, but they really do not give a damn about ARM support. It’s more a user community driven sideshow as far as Slackware is concerned. Maybe that will change as ARM now outnumbers Intel by a lot (on chip count, not $$$ sold total). Then Void is still teething a bit more than I like and Puppy is just too kitsch and odd for me to want to embed in their devo world.
So that’s where I am today.
Comments, nudges, and rock tossing welcome ;-) Better to thrash out a tech direction before you spend a year learning everything needed to show it was the wrong direction ;-)
@EMSmith according to my grandson:
Isaac Sharrow (18.10.2019 15:02):
The problem is mostly updating and managing the base Linux Kernel options. He’d be strung up by having to wait for RPF to update the kernel themselves, which also kills being able to go after Kernel-level security options, and a bunh of low-level controls.
pg (18.10.2019 15:08):
I think I almost understand. you are talking about the base programs that instruct the machine code
Isaac Sharrow (18.10.2019 15:09):
Yup. That’s the most important layer when working with DIY Linux work.
Isaac Sharrow (18.10.2019 15:10):
Here we goooo! This looks more open, bruteforce community opened like the Pi, but on flat-out better hardware. Looks like it’s on the RockPi 4, We’ve got a busted open driver for Rockchip integrated MALI GPUs!
pg (18.10.2019 15:11):
RockPi is the $60 SBC that smith likes
pg (18.10.2019 15:11):
when he can get it to work
Isaac Sharrow (18.10.2019 15:11):
It’s got a open source U-Boot port! Yup, this is the place to start. RockPi 4 with the RK3399 processor. <3
pg (18.10.2019 15:12):
It has USB3 ports
Isaac Sharrow (18.10.2019 15:12):
And M.2 support (though you need a special case to use it right, grr…)
pg (18.10.2019 15:12):
and 2×4 processors that really cook 64bit parallel
Isaac Sharrow (18.10.2019 15:16):
This might be the next place to look. Then again, I said the same thing about the Raspberry Pi so maybe my "Expert Opinion" isn't as expert as it might sound.
an extra 2 bits worth of opinion…:-) pg
The above was the result of a conversation about the problems of OS management of the Broadcom – Linux interface with the Kernal
“Must’ve bought it from allnetchina,com another Huawei owned company”. Would you believe?..pg.
@P.G.:
Well, I’m on the Rock64 right now. Happily purged of systemD. I was considerably peeved when I finally figured out that the BOARD was fine and it was just an Ubuntu / SystemD crappy behaviour causing it to not boot and give a black screen.
The general structure of software on an SBC is that it starts with the kernel. This is the thing that knows how to provide all the core operating system functions. Open a file. Load a program. Make the hard disk spin up. Then there is a “DTB” or Device Tree Bundle? It has all the proprietary and otherwise bits of program to make your particular hardware work. Things like a specific network card driver for your network card, or the code to make your mouse work.
That’s what gets you to a basically up and functioning bit of hardware. But you still can’t do anything with it. You need a “userland”. All the programs you know and use…
Note: I’m skipping some details in here. Such as that there is a scrap of code called a boot loader that just picks up the kernel and stuffs it into memory and sets it running, and there’s often an “initial ram disk” image that is a stripped down version of some of the code that gets loaded into RAM during that boot phase and acts like a disk-in-memory before you have the operating system up to where it can deal with real disks… The actual “boot process” can be strange and wondrous and varies from machine to machine. For the Raspberry Pi, one of the complaints about it is that the boot process is a binary blob that runs in the Graphics Processor portion not the CPU. I’m OK with that, though it IS a potential security attack surface. (BUT if someone at the vendor is compromising your boot code, you are already screwed).
So once the boot loader has loaded the initram disk and kernel it then hands control over and the kernel loads up some userland. At that point it starts all the background daemons (printer spool drivers and NFS servers / clients and such) and presents you with a login prompt.
The problem I was interpreting as a dead board was at that point where the kernel is booting, and trying to attach disks. An /etc/fstab entry ending in “noauto 0 0” ought to have it do nothing. The “0 0” says to not do an fsck (disk check) on it, and the “noauto” says to NOT automatically mount it at boot time. On Ubuntu, this failed in how systemD interpreted fstab and resulted in a black screen and no response to the keybaord. IMHO a flat out horrible bug, all caused by SystemD deciding to take over too much turf and doing it wrong. (This was “fixed” in Armbian which tends to confirm my opinion).
As a consequence of this having hit me on at least 3 boards over a couple of years (and my mistakenly interpreting this as the board dying) all of which I’ve now recovered (AFTER talking dirt about them in postings here – my bad) I’ve now banished UBUNTU from usage in my shop. IF I’m running a SystemD afflicted OS it will be Armbian.
That said, I’m presently exploring package management on Slackware on the Rock64. Slackware isn’t as “slick” as Debian / Devuan wiht a more retro feel, but it seems to be fairly rock solid and it isn’t doing stupid stuff.
@Per Huawei:
You KNOW China is a communist dictatorship.
You KNOW the government owns most of any significant company.
You KNOW China has a big spying / dirty tricks focus.
You KNOW the general “economic culture” of China is “screw you to enrich me”.
You KNOW there has been a long history of adulterated products and virus implanted devices.
Given all that, why on earth would anyone TRUST a Chinese computing product? I don’t.
Part of my preference for the Odroid family of products is that they are made in S. Korea. You can generally trust the Koreans and they do good design work.
Now I do have some “Made in China” boards. Pine touts itself as designed in Silicon Valley made in Silicon China or some such. For them, I’m depending on the designers to keep the fab honest and on the way these boards are mostly going “nowhere of interest” to make them a low priority target for Dirty Tricks. Were I doing anything of consequence, I’d either NOT use a Made In China board or it would be strongly Guard Dogged by an IDS / IPS system logging network activity It is also the case that these boards are simple enough that an inspection of them ought to show up any hardware Dirty Tricks and then I put my own software layer on them. (The Bugger there is that the vendor often supplies the DTB as a binary blob – which is why the hard core folks decry binary blobs… including the one on the R. Pi…)
So essentially I’m taking a “measured risk” with the Pine64, Rock64, and RockPro64. But since I’m mostly not doing anything interesting on them, they are isolated inside a lab network, and I have no PC MicroSoft stuff running (THE major target for infective agents) .I’m OK with that.
But, just to be clear, were I doing ANYTHING that I didn’t want to risk having China know about, or having a potential “back door” in that network, I’d be doing it on the Odroid XU4 or C2 or N2 and not on anything made in China.
“It isn’t paranoia when you KNOW they are out to get you”.
And we know China is out to get us…
@Slackware Topic:
I did a “dd of=/SWAPFILE” to make a 1 GB swap file. Then could not get much response from any other window. The “htop” window kept updating, but with long pauses. EVENTUALLY, I could get some response in another window, but with great delay and sloth.
My interpretation of this? Either their “preemptive scheduler” isn’t preempting worth a damn, or once you launch something that saturates the disk (uSD card) data channel it blocks other traffic until done. (Sun tended to do that with network packets. Too small an inter packet gap and it would block other vendor traffic. Gave them great stats on network performance but didn’t play well with others…)
It isn’t a horrible thing, just something to be aware of. Launch a big multiple GB “dd” and you better go get coffee because you are not going to get much done until it completes.
So, OK, I need to test that with “nice” in front of it (is it just a priority thing?) and doing it NOT as root (is it a root priority thing?) and then try some non-disk-bound processes (is it a disk driver thing?). But, in general, I can live with that. I’m prone to just launching some big ‘dd’ copy and coming back when it is done anyway.
So a little “sand in the gears” but still a heck of a lot better than a black screen and nothing ;-)
It may also be part of why, in general, Slackware just seems ‘snappier”. It isn’t chopping everything up into swapped slices with a lot of preemption time and code swap time, but instead just running the foreground code for a good long shot.
That IS one of the perpetual arguments about schedules. How much to just “get the job done” and how much to “swap between processes a lot so it’s ‘fair’ to everyone, even if slower.”
I’m going to be putting my Tech Time into 2 things for a while. 1) Getting decent at systems admin on Slackware. and 2) getting Gentoo running and using it on some board.
As those are both fairly tech heavy distributions / tasks, it won’t be quick. Yes, I’ve run both (briefly) at various times in the past, but never as a major daily driver. More a “build it / try it / move on” experiment without a lot of systems admin post build. So I have some work to do.
In general, so far, I find Slackware quite acceptable and better than anything with SystemD in it.