I have a BUNCH of links open in my XU4 FireFox browser and it’s chewing about 100% of CPU (it can go higher than 100% as it is 8 cores, but still, I’m getting sporadic pauses). So… I’m going through turning some into bookmarks and just deleting other tabs. I’ve decided to aggregate the Gentoo tabs into this posting to make them easy to reach from any of my machines.
If you are not interested in the XU4, Gentoo, or how to install Linux on an ARM system, you likely don’t care about any of this.
It is in no particular order. Just grabbing the link and stuffing it here for the future. “Someday” I’m likely to go to Gentoo on the XU4 simply because, while a PITA to install, it is a SystemD(emented) free port. But not, I think, today; so saving the links is it.
Generic “where to get OS images” link at Odroid:
https://wiki.odroid.com/odroid-xu4/os_images/os_images
Not Gentoo, but Arch, who have decided to BOHICA and let SystemD(eviant) have it’s why with them:
https://archlinuxarm.org/platforms/armv7/samsung/odroid-xu4
The Arch ARM repository:
http://il.us.mirror.archlinuxarm.org/os/
Nice article on installing Gentoo:
https://programmingmiscellany.wordpress.com/rust-never-sleeps/the-odroid-xu4-with-a-new-os/
Installing Gentoo base system:
https://wiki.gentoo.org/wiki/Handbook:AMD64/Installation/Base
The OpenRC init system (a Gentoo alternative to both SystemD and RC based inits):
https://wiki.gentoo.org/wiki/OpenRC
Some Gentoo XU4 bits and with emphasis on formatting partitions:
https://wiki.gentoo.org/wiki/Odroid-XU4
https://wiki.gentoo.org/wiki/Odroid-XU4#Formatting_the_partitions
How configuring Gentoo Repos works:
https://wiki.gentoo.org/wiki//etc/portage/repos.conf
The Stage 3 files:
http://distfiles.gentoo.org/releases/arm/autobuilds/current-stage3-armv6j_hardfp/
What to do if your emerge keyserver refresh fails:
https://forums.gentoo.org/viewtopic-t-1083474.html
Bugbase on a keyserver related bug:
https://bugs.gentoo.org/647696
Sync failure discussion:
https://forums.gentoo.org/viewtopic-p-8186000.html#8186000
A related, but different, derivative of Gentoo, Funtoo. Design goal is removal of some Gentoo PITA:
https://www.funtoo.org/ODROID-XU4_Installation
The Funtoo download (and what I’m more likely to try next when “someday” I return to doing a Gentoo / Funtoo on XU4):
Musings from the Chiefio 
Well, pruning tabs didn’t do anything. I discovered the English Dictionary wasn’t working, so went to the addons page:
https://addons.mozilla.org/en-US/firefox/addon/us-english-dictionary/
to do a remove / reinstall. It is now spinning endlessly on the remove step.
So, I figure something made the Dictionary barf, and now it doesn’t work, and isn’t “installed” enough to be removed. Next I’ll try just crashing the de-install and then re-installing. IF that doesn’t work, I’m a bit hosed. This release is out of support and the repositories have moved off to archives “somewhere”, so do remove / reinstall Firefox in total I’ll need to figure out where all the repositories have gone off to and change my /etc/apt/sources.list accordingly.
Sigh. Not what I had in mind for this morning. Maybe I’ll just boot up on a different system…
Well, that was interesting…
I crashed the “remove” page and instead installed a British English Dictionary. Then the American English Dictionary would not let me chose “remove” again (even though it showed only the “remove” button). After shutting down FireFox, and then relaunching it, spelling checks once again work (though of the British flavour and colour) and I am again offered the opportunity to install the American English Dictionary. So something got cleaned up
CPU “spin” is down some, but still higher than I think proper at about 75% of a core while doing nothing. Oh Well. Good enough.
Speaking of ARM, I have seen reports that nVidia is going to buy them.
Yeah, but….
ARM corporate doesn’t make chips. They license instruction sets and core designs. So oodles of folks have existing licenses and roll their own. Samsung, Apple, and many more. Those licenses have persistance.
So what is really being bought is royalties from old licenses, and the right to design new instruction sets going forward.
A bit different from, say, buying Intel with design, fab, inventory, distribution, etc.
Then realize that ARM, nominally RISC, has been adding in complex instructions like pipelining as Intel patents expired. There isn’t much left down that road as Moore’s Law is winding down and most interesting instruction ideas are already known (and many expired patents).
For over a decade now, most speed gain has come from more cores. ARM is now at 64 bit pipelined multicore (up to 8). There are effectivly increasingly hard limits that start biting hard at 64 bit, 8 cores, and 3 Ghz.
Beyond 64 bit, packing and unpacking bytes takes away any math speed gains.
Beyond 8 cores, few problems can effectively use all cores (at 2 k cores it is hard for even highly parallel codes for highly parallel problems to use all the machine)
Physics is limiting at about 3 GHz and power needed goes way high.
Physics is limiting at current feature nm width. Essentially, masking smaller requires using X-rays and shorter that just pass through the mask…
So not much more that can be done to advance things on that front. Thus the big move to using CUDA Cores in nVidea video processors and massively parallel coding practices to reach the hard parallel limits at 2k to 4 k cores.
My best guess is the present ARM owner (Softbank?) realized the super growth of the past was over, while nVidea is growing fast in the massivly parallel space (computer vision, self driving cars) and figured adding some ARM bits to their CUDA cores designs might be useful.
https://chiefio.wordpress.com/2015/10/23/amdahls-law-gustafsons-robots-and-jobs/
FWIW, I’m posting this comment from the RockPro64 running Slackware. I just did an update of it (as described in the comments here: https://chiefio.wordpress.com/2019/10/21/the-first-few-days-of-slack-ware/#comment-118844 ) and FireFox is nicely running a video with good audio.
As almost all I need from my general purpose desktop is a working browser with sound and enough computes to run video, I think I’m set.
The “Devuan Uplift” on the Odroid XU4 has just gotten too far out of sync with support. First off, the Jessie to Devuan uplift script is now deleted. Second, Armbian (the base Jessie I used) has archived the sources.list target (and in modest looking, I could not find a way to download it to my own sources server nor any alternative other than pointing at generic Debian and risking killing off bits of Armbian / Devuan that matter…) So I’ve run without updates for about a year now, and some things are starting to be an issue (i.e. not being able to install new applications).
So at least until I’m able to put some other OS on the Odroid XU4, it is set aside. I’m going to be moving “stuff” from it to the RockPro64 since now it seems to do what all I want. (Not a real surprise. Usually when you buy “The Latest Hot Board” it takes about a year for the software to be all working right and bugs ironed out.
Hopefully I’ll be happy with this Slackware release. I do need to spend a bit of time getting my Slackware Admin skills up. It is BSD style RC based init. It’s been about a decade since I did much with one of those. OTOH, with about 20 years+ of maintaining it professionally, I’m pretty sure all that stuff is still in the brain somewhere ;-) Biggest issues is the Slackware specific stuff like “slackpkg” for slackpkg update and slackpkg install-new.
So it goes…
Slackware has been around “since the beginning” and has been prone to NOT embracing all sorts of gratuitous and IMHO stupid changes from BSD roots. I’m good with that for most of my uses.
It does have some limits that can annoy. It isn’t particularly NOOB friendly. The boot process is the older slower one thing at time version (OTOH you can watch and see when and where a boot fails in real time…)
Then there’s stuff I’m not sure about yet. Does it have all the interesting file systems I like to play with? Will it have extra problems with newer versions of applications as SystemD forces changes n them? We’ll see. For now, it has all I need in a Daily Driver desktop. GIMP for image creation. Working browser with good sound. Functioning (AND STABLE) administration and file storage.
So that’s where I am this morning.
I’m looking at trying NetBSD on the Rock64 / Orange Pi One for various infrastructure uses. At present, the Orange Pi One file server is running Armbian, but the way Armbian just deep sixes support for a given release is a bit problematic. It means I’m either an orphan by surprise, or I must watch for the right moment, make a duplicate of the sources.list targets before they evaporate, and / or stay in sync with their constantly moving versions. (Which is problematic as it is a big workload, updates risk system failure, and then for some boards support may just end.)
So IMHO Armbian is fine for first use for a year or two, but then you either need to be prepared to suddenly go “Stand Alone” or accept that you may be suddenly unable to add applications or do updates. And your board may be left an orphan once they lose interest in it.
As they are “embedded” oriented, that’s reasonable. Vendors need support for a couple of years, then they are making new products on new boards and have moved on. IF the vendor does LTS Long Term Support on a product, they have a LTS team and have downloaded a set of sources and have their own sources.list target.
Basically, Armbian isn’t “desktop oriented” and I am. I can live with that for a couple of years, but not for decade+, and I use equipment for decades…
Anyway, enough of a rant. On to more productive things.
Hit my first issue. The “old” home directory account on the XU4 was successfully added to Slackware on the RockPro64 (via the wonderfully simple old school way of just editing /etc/passwd and /etc/group ;-) and the file system mounted.
Logged in and everything was there, even my desktop looked right. Launched FireFox…
Newer Firefox on the RockPro64 / Slackware informed me it used a new “Profile” system so all my old settings, bookmarks, etc. needed to be moved from the old to the new. “All” I needed to do was make a FaireFox account…
Well, being unfond of making accounts all over and telling everyone all about me and getting SPAM email from the forever and being sold to vendors and…
There’s an alternative, which is to reboot on the XU4 and dump my bookmarks, and open tabs URLs and then reload them here. OK, not too bad. Except I just put the XU4 back in the shipping box on the shelf.
Yet Another Someday Task… For now, I’m using the local RockPro64 maintenance account.
FWIW, I typically make 2 accounts on any machine / board. My “Daily Driver” general use accunt, and a “Boot and maintenance and emergency” account. That way if I screw up something in my Daily Driver account and can’t log in, I can log in as the “other me” and fix it. This also lets me avoid all the security hassles of trying to log in as root directly (some systems block this, others only let you do it in single user CLI Command Line Interface mode, etc.)
Both accounts are fully functional and both are configured about the same. Just one gets most of the files, clutter, etc.
So, OK, not a big problem. just something to note: FireFox has backwards compatibility barriers somewhere in the last year or so. Welcome to Dependency Hell (even if only a minor level of hell…) https://chiefio.wordpress.com/2019/11/01/linux-dying-in-dependency-hell/
FWIW, I don’t have a lot of stuff in bookmarks or open tabs on the other account. I moved most of what I care about to this Gentoo posting. The rest I can fairly easily get back. So no big. Just a minor carbuncle on the rump of life…
Oh, and while I’m here:
That Amdahl’s Law graph above, how to read it:
Typically in a normies mix of workload, about 50% is suited to parallel processing. So look at the 50% line (light blue at the bottom). That’s saying you can cut run time in half if you use abut 8 cores, then you get no faster. That is why most desktop machines have at most 4 to 8 cores. Beyond that the typical user gets nothing more to show for the money.
The half that does not run parallel now limits completion time.
At the top, the green line, is showing that for an incredibly parallel computing task where 95% of it can be done in parallel, you can get a 20 times speedup at about 2000 cores, then you get nothing more. Supercomputers with 4000 or 8000 cores are really only “better” in that they can run 2 or 4 such jobs at the same time for different people (or can run tailored test cases very fast to claim superior performance over the competition in gross computes…)
The 5% that does not run parallel now limits completion time.
So consider a climate model with 16,000 cells. At about 2000 cores, you are maxed out. Why? Because once your run of cell 1 computes, it is I/O bound moving those results to where they need to be while you compute cell 2, 3, 4, etc. values (that then get I/O bound in the write step).
about
Now it MIGHT be possible with very careful code design to make a system where each cell was mapped to a single core with local data store, and get 16,000 cores to all hummm together, but that’s way hard. Needs things like special data prefetch to move data from the database to each core cache storage, then careful handling of any subsequent write steps and more. Even then, it is unlikely you can avoid a fair amount of core idle time during I/O operations. Your gain over the case of 2000 cores and I/O processors handling data flow will likely be small (and very hard to attain).
Essentially, you end up custom designing the computer and the data flow to match the problem, should you want to effectively use more than 2,000 to 4,000 cores. Though really, just to get to using 2000 cores you already did a lot of that. “Massively Parallel is Massively Hard. -E.M.Smith”
While we’re not up against the wall just yet, and we may still have a “Miracle Happens” moment with some new bizarre technology, in reality we’ve got about one last Double Of Performance before it’s just not worth it anymore at the hardware level. At that point (and not likely before it), the focus will turn to all the horrible software bloat that’s gotten into things in the last couple of decades. Heck, folks might even start writing assembler again ;-)
To some extent this is already happening. There are special languages and compilers being made to improve parallel execution of codes. More use of Graphics Vector Processors for math bits in parallel parts like rendering and big math problems. OS tweaks to make parallel threads easier and let more of them run. But it is going slowly in most cases.
So, for example, on this RockPro64, in the browser, with a video running in another panel, I’m using about 1/2 of the 6 cores. It looks to me like about 3 to 4 cores of load but being spread arround. More cores will not give me any more performance on these tasks. Htop shows 255 threads in existence (many of them for the OS itself). So I’ve got a potential for 255 parallel tasks, but not filling up the box anyway so more cores not going to do much. Only 3 running at the moment, most waiting for some work to do.
Found an interesting discussion of Slackware on the XU4. I’ll add the link here later if I can. Essentially they did it via putting the Slackware userland on top of an Ubuntu kernel, modules, and libraries. I’ll have to ponder that a bit.
It ought to work OK, and is less work than doing a full on Gentoo compile (as Gentoo is all source build, you spend a day or so doing compiles…). I’ve done other userland swaps on other systems, and it isn’t that hard. Biggest issue is just the risk of some Dependency Hell from different levels of libraries in kernel vs userland, but that’s usually OK if the releases are at all close in age.
I find Gentoo (or Funtoo a more user friendly derivative) interesting as it IS a source build so you know it all works together. (BUT if it fails in the compiling steps it is up to you to fix it…). OTOH, Slackware is built binaries and I’ve already got it running on 3 or 4 different boards (but it suffers from being a bit out of date a lot on some applications and being a bit retro in look and feel at times).
Decisions decisions…
For now, I’m mostly going to work on moving my old Jessie Firefox to the new Slackware version on the RockPro64, and having it be my Daily Driver for a while.
I’ll then archive the Armbian-uplift-to-Devuan-Jessie image from the XU4 and at that time see which port works for me. Probably do a “First one to work wins” with both of them.
FWIW, on ARM boards, it is looking like Raspberry Pi gets the most support and ports. Other boards are a bit more “catch as catch can” and spotty. Armbian has the most breadth of support, but they only keep a couple of releases “up” and live for their source.list targets at any one time. You are expected to keep up with the constant river of change… No “build it and forget it”.
Neither Slackware nor Gentoo (Funtoo) have a great depth with ARM boards, but both have working ports to major CPU types. Around the edges of odd boards, it is more “roll your own” and a bit technical. Mix in that Odroid have a particularly bothersome boot process and that means they get less love than others. Sad, since IMHO the Odroid Hardware is some of the best thought out and have proper heat removal built in. Oh Well.
So, in summary:
I’m still doing the “wander in the woods” of OS / release candidates that will let me avoid SystemD while also being relatively low support workload. For things that are modest use (i.e. very little admin on them so SystemD interaction is minimal) I’m still using Armbian, but staying in sync with release levels at least once a year. I also need to put a bit more priority on making a source.list archive locally so as to not have the archived release death bite me again. (I’ve done it before and there’s an article up about it, my only “problem” was that I didn’t make one for Armbian:
https://chiefio.wordpress.com/2017/10/02/debian-devuan-raspbian-repositories/ )
I’m mostly going to run Slackware, for now, on my non-Raspberry and non-Armbian systems. We’ll see if I can get a Gentoo build on some other board to work without too much pain and suffering, then I’ll take on the Odroid boot differences.
The “good news” is that I seem to be centering in on Slackware and Gentoo (Funtoo) as the most likely candidates for non-Devuan systems. As much as I like Devuan, they just are not interested in providing systems for more than a couple of makers of boards at this time.
This looks promising, if still alive:
http://mirrors.aptalaska.net/slackware/schu/slackwarearm/README.md
Well, I’m moved onto the RockPro64 / Slackware now. Had to dump / restore bookmarks from the XU4 as this Firefox is not compatible. Also made a “private” copy of URLs that were open in tabs but not bookmarked to “save state”. I’ve likely lost other state information (history / cookies) for sites that depend on them to let me log on, but “We’ll see”.
This is a faster experience than the XU4, some of that the 2 faster cores, but more of it, I think, the slim software. Slackware has always been a bit well optimized and low bloat.
If this experience goes well, then I’ll put a Slackware on the XU4 and call it done. If it goes badly (i.e. if I find myself not fond of Slackware or can’t load needed software) then I’ll try a Gentoo install.
Looking over Devuan, it looks like maybe The Newest Release has shipped the x86 versions but that the ARM boards might just be following slowly and later. Maybe an XU4 port will show up after all. But for now, the XU4 will just sit in the box a while.
Both Slackware and Gentoo on the XU4 have you save a kernel, modules, and headers from some other build and then glue on their userland (along with keeping the Odroid specific boot binary / disk partition stuff). I’m OK with that, having done it before, but would really rather not have a Franken System.
Oh Well.
While I really like the XU4 as they have some of the most computes / $$ and a nice stable 32 bit system with good heat control: fighting with getting a non-systemD OS for it is too much time and trouble. 64 Bit software has matured enough on ARM that it is no longer an issue, and other boards have simpler boot process so get more “load and go” support. I may just have to “move on”. Make it a general use desktop or some dedicated infrastructure thing.
I’d intended to make it the hart of a 32 bit cluster, but to do things like distributed compiles requires the same release level of libraries on all parts, and that’s just not going to happen in a heterogeneous environment of different boards and it. Compare the Rock64 and RockPro64 where both easily installed and booted Slackware. So 10 fast cores all the same OS, no grief. Or the Raspberry Pi stack where I’ve got 16 cores, all the same software (and several choices of it).
So “it is what it is”. As of now, the XU4 is on the bench and the RockPro64 has had software maturity catch up to where it’s very nice. So it’s the Daily Driver and candidate for Build Cluster cores. I do need to find out if the “missing bits” from first release (that caused me to set it aside for a while) are really all there now. Python. SQL. Gimp. etc. and working properly. IIRC, I was having issues with plots in Python from the database…. so checking that needs doing.
But as a general desktop, browser (modulo the link / bookmark / history / profile conversion), file system and workstation, it looks like it’s pretty good.
And no sooner said that then found out GIMP isn’t working:
bash-5.0$ gimp
gimp: error while loading shared libraries: libbabl-0.1.so.0: cannot open shared object file: No such file or directory
So some libraries missing or misplaced. Guess I’ll try a GIMP reinstall…
Sadly, it looks like ARM systems are still treated as 2nd class citizens. Gentoo has a Stage 3 tarball, but installation instructions are lacking:
https://www.gentoo.org/downloads/
https://wiki.gentoo.org/wiki/Handbook:Main_Page
E.M. Smith added on 19 September 2020 at 7:10 pm GMT:
[….] on ARM boards, it is looking like Raspberry Pi gets the most support and ports. Other boards are a bit more “catch as catch can” and spotty. Armbian has the most breadth of support, but they only keep a couple of releases “up” and live for their source.list targets at any one time. You are expected to keep up with the constant river of change… No “build it and forget it”.
I am expected!? Well! [Expletives deleted]! That’s about as potent a disqualifier for software for my personal use as I’m ever faced with, aside from outright unsuitability for the tasks to be performed.
In certain noncomputational Chiefio topics, I’m among those alluding to my “comorbidities”, so my remaining life-time is likely too short to have any of it wasted by source-code management practices as, ummm, miserly as Armbian‘s. What’s the problem that its people think they’re solving? Don’t they understand that they’re risking making their distro “too bothersome” to survive?
Hmmm… reading this over:
https://www.funtoo.org/ODROID-XU4_Installation
It has you prserve the boot sectors, kernel, headers and , modules. Then place them in an unpacked Funtoo image.
This leads me to think I could just unpack the Funtoo into selected partion segments and overlay it on an existing working system…. as I’ve done with 32 and 64 bit Pi O/S on one chip… I think I’ll try that. Later.
@Compugator:
They are mostly solving the professional embedded systems maker issues, not the home desktop on ARM hobby user issues. In that context their choices make sense. Not the ones I’d prefer, but right for that market.
As ARM starts intruding into the desktop space, that will change. But only now is it starting outside the hobby space (Apple and a low end laptop or 2).
They make ports to as many SBCs as possible, then hand it to various device maker staff who grab their own copy and sources and customize for their product LTS. Those are the folks paying the bills.
So some company with a custom router based on the Jessie port will have copied the stuff for archive for their LTS product needs, but the developer staff will be on current or devo … in the river of change.
I saw the notice that Jessie was moving to archive, but ASSUMED that would be an open online archive. Not telling me to archive my own, too. My bad, not theirs.
Well, progress, even if not fully satisfied.
I now have a working Funtoo install on the XU4 via a slight kludge. The SD chip is now dual boot. So I can boot the old Armbian Jessie uplifted to Devuan, or boot the fresh Funtoo install. Or “chroot” into Funtoo.
I did a modification on the Funtoo page idea. Cleared the third partition on my chip (really checked it was empty) and changed the lable to SD_Funtoo, then unxz uncompressed and tar xvf the tar file. At that time, it has no /lib/modules nor /lib/firmware, so copied them from the running system. Also copied over /boot just in case. Edited the new fstab and went to chroot /SD/Funtoo where I had mounted it. Then you can set root passwd and do other config stuff.
Finally, added a line to each boot.ini setting rootfs to that partition and rebooted.
Came up live.
I’ll put up more details and files later.
It is a modestly thin system out of the box. Very fast, but missing bits I want. Like gparted, X windows, etc. Funtoo uses yet another package system, so next for me is to learn how to add missing packages and turn it from CLI only into a desktop.
For now I’m happy to have it running.
As I can chroot into it in a terminal window, I can boot the full graphical Armbian / Devuan OS then in a window work on adding packages.
So off to learn Funtoo package management…
Probably ought to put all this in a new Funtoo – Not! posting…
So in the chroot, I’m trying to update / upgrade and SW install desired bits. It isn’t working. Of course, “everything you know is wrong”, again, because Yet Another Bright Idea Guy decided to change things yet again. OK…
In addition to the already byzantine Gentoo “emerge” system (with EVERYTHING configurable but not set right out of the box) it has layered on more stuff. “ego” and “meta-repo” and using “git” for packages and having meta-packages called “kits” (that used to be called “shards” that…).
But hey, the “directions” say to just run “ego sync” and all will be fine…
Which produces the Oh So Useful log:
XU4_Funtoo /tmp # cat ego-traceback-3711.txt 'release_defs'Traceback (most recent call last): File "/usr/bin/ego", line 118, in EgoModule.run_ego_module(action, econfig, args, VERSION) File "/usr/share/ego/python/ego/module.py", line 103, in run_ego_module ego_module(*args) File "/usr/share/ego/python/ego/module.py", line 88, in __call__ self.handle() File "/usr/share/ego/modules/sync.ego", line 452, in handle self.sync_meta_repo_and_kits() File "/usr/share/ego/modules/sync.ego", line 412, in sync_meta_repo_and_kits EgoModule.run_ego_module('profile', self.config, ['update']) File "/usr/share/ego/python/ego/module.py", line 103, in run_ego_module ego_module(*args) File "/usr/share/ego/modules/profile.ego", line 313, in __call__ super().__call__(*args) File "/usr/share/ego/python/ego/module.py", line 88, in __call__ self.handle() File "/usr/share/ego/modules/profile.ego", line 294, in handle handler() File "/usr/share/ego/modules/profile.ego", line 130, in handle_write self.tree.write(self.config, outfile) File "/usr/share/ego/python/ego/profile.py", line 462, in write for kit in self.config.all_kit_names_in_release: File "/usr/share/ego/python/ego/config.py", line 97, in all_kit_names_in_release return self.kit_info_metadata["release_defs"][self.release].keys() KeyError: 'release_defs' XU4_Funtoo /tmp #OK….
So what’s the problem? What to fix? Web searches are unenlightening…
One says to set the permissions on a directory to what they already are. Another has directions for testers interested in trying this new meta-kit thing from years ago. Etc.
Maybe it’s just my low exposure to emerge (and the new “ego” in Funtoo), but I really get the feeling some parts are missing… and steps to make them go…
AND, I’m not even sure where to begin.
So I’m going to try semi-randomly whacking at bits of this to see if I can get more clue as to what’s missing / wrong / not configured / broken / whatever…
BTW, doing “emerge –sync” (the Gentoo way) seems to give the same error file.
Don’t know if this fixes it, or not. It claims to be how you set up to use meta-repo; yet the installed system is already trying to use it as though it thinks it is already installed.
I think I’ll clone the chip (about a half hour) then try this;
https://www.funtoo.org/Funtoo_Kits
Using Meta-Repo
To use meta-repo as your meta-repository for Funtoo Linux, perform the following steps on your Funtoo Linux system, as root:
Well, tried to do that and it looks like DNS resolution isn’t working (so did nslookup in another panel and tried it via number, no joy.)
So also hung up on some cert stuff too.
But ah, the joys of a chroot system. Swapping back to the parent system and doing the git there:
Gives me a meta-repo in the chroot system:
So onward and hopefully upward…
Continuing that process, back on the parent:
and then:
So now I’m going back onto the chroot Funtoo to see if NOW I can get it to work (after figuring ut why the networking isn’t in the chroot?…)
Well, at least now “ego sync” didn’t outright die, but it does look like I need to do the rest of these steps from above too:
As running “ego sync” said:
Meaning my edit of /etc/resolveconf.conf to add the local name server was not enough. Perhaps after rebooting as a live system instead of a chroot….
and:
So that’s some progress. Now on to those file swaps and a try as a running system…
Hmmm… /usr/share/portage/config has no repos.conf file to remove. OK I guess.
The other stuff done:
Ought to just leave name resolution.
On the off chance I’m just not passing it right in the chown partition, I’m going to boot as Funtoo and try it in a live system. (Which at present has no X-windows, browser, etc. so it will be a while before I can post status updates like this…)
Had to do an “rcupdate add dhcpcd default”
As listed here:
https://www.funtoo.org/Install/Network
Then reboot.
Now “ego sync” looks like it is running fine!
It slso looks like it may take a few hours to sync then update world, and capturing that output one terminal CLI land isn’t going to happen, so not much to update here for a while.
But if looks like I’m over the bootstrap hump!
Well, the update step:
emerge -auDN @world
Looks to be running fine. At package 25 out of 34 updated ATM.
Mostly Python 2.7 stuff, is seems. Lots of source compiles.
When that completes, I’ll grab another backup copy, then proceed with installing X-windows and other stuff.
https://www.funtoo.org/X_Window_System
https://www.funtoo.org/LXQt
Assuming all that works, next is to take my Debian software install script and make a Funtoo version with all my preferred bits. After checking what all gets auto installed as part of a “kit”.
Since Firefox will be one of them, and the source build can take hours (days?) on smaller machines, I’ll need to do some tuning first. Things like putting the build scratch areas on real disk and setting compile flags to 8 cores…
The really good thing is that I’m past the undocumented ARM XU4 not like a PC parts, mostly. From here on out, the documented procedures ought to help more than hurt.
So the @world update completed, backup in progress.
Looks like about 2.5 hours
Attempting to add x11 failed on the Funtoo guide being 100% PCI bus PC. This page looks to have XU4 video driver advice:
https://wiki.gentoo.org/wiki/Odroid-XU4#Copy_X11_configuration_files
Hey, I think I may have found where Devuan archived the Jessie stuff:
http://archive.devuan.org/merged/
OK, One success! I now have a local Devuan Jessie repository. So no updates happening, but at least I can install new applications.
I made a copy of the archived sources using a wget command (in a one line script named “getdevuanarch”)
Looking inside it has .gz files, so change “deb” to “deb-src” in your /etc/apt/sources.list file.
And change from an http: to file: at the head end (remember that you don’t put in the full path as “jessie” is found via the release level and “dists” is assumed):
Note that there are not updates nor security patches being made anymore, and backports was not there (on the presumption it has all been rolled in?) so those gave errors and I commented them out. My bad.
It’s obvious they are not there when you bother to do an ls in the dists directory…
And now update and upgrade work without error (so I assume adding a package will work…)
Now the good news is it looks like I’d done an update / upgrade after the archive stabilized so I wasn’t out of date with the final image. Just could not install more packages. Now I can.
That’s about 1/2 my “issues” on the XU4 right there. Pressure reduced to find an alternative!
Now I’m going back to a more leisurely pace of trying to get X and LXqt working on Funtoo.
Pingback: Slackware on RockPro64 – Progress | Musings from the Chiefio
There is a prebuilt Gentoo 64 bit image for the Raspberry Pi M3:
https://github.com/sakaki-/gentoo-on-rpi-64bit
Complete w/ working xWindows, browser, etc.
I’m going to give it a tty “soon”. There are also directions for making a cross build environment for ARM systems.
https://wiki.gentoo.org/wiki/Raspberry_Pi
Also, every board I have has had a working Gentoo ported to it:
https://wiki.gentoo.org/wiki/Embedded_systems/ARM_hardware_list
Part of the problem with Gentoo on arm is the slow build speed of everything from sources. With a pre-built Pi version, I can bootstrap off of my 2 x RPiM3 boards with 8 cores of distcc compile power. Then each subsequent board built, adds more cores. I can break out ofvyhe “configuing Xorg is a pain” trap…
There are also facilities to compile the 32 bit v7 code on the 64 bit v8 OS. Though I’d be very tempted to just boot the available 32 bit Gentoo on the Pi and avoid those cross compile issues.
So sometime tonight or tomorrow I’ll be test driving this Gentoo prebuilt image and trying out some distcc distributed builds stuff just after.
The directions for Gentoo on the XU4 look pretty clean, and it is 8 x 32 bit cores, so would make a good 32 bit distcc main node, with the 2 x PiM2 as added compile nodes. And the Odroid C1. 20 total cores ought to build quick. Then an SD swap on the 2 x PiM3 to run them 32 bit would make a 28 core cluster right quick.
Similarly, using the Rock64, RockPro64, 2 x Orange Pi One, Pine 64, Odroud C2, and running a 64 bit OS on the 2 x PiM3 would give a 34 core 64 bit cluster.
I’d have to clean off my desktop to have them all set up at once, though ;-)
It will take a while to do all that, and Gentoo is a technically challenging culture… even with one build a day, it would take 2 weeks to make them all. I think I’ll start with just the Pi stack as that Devuan 1.0 is out of date…
Well, that was quick. I’m typing this on my Pi M3 running the full configured Gentoo.
It unpacks to about 13.x GB (!) so you need at least a 16 GB chip.
The wifi and keyboard come set to UK. It does present a nice configuration panel at first boot where you can change the WiFi domain. I’ve not yet changed the keyboard…
It is fairly fast! The Pi is actually quite usable for browsing (Chromium installed already).
All in all, quite painless.
With gparted open showing the disk layouts, htop running, and Chromium open with 3 tabs, memory usage is just 704 MB. and CPU is running about 1/4 nicely spread over all four CPUs.
OK, I’m sold on Gentoo on the Raspberry Pi. Next up will be converting my other M3, and then getting a port done for the M2 and even the B+ (eventually….)
Then, over the next few weeks, as the mood hits me, I’ll be trying to convert each of my other boards. It is VERY nice to know the same Linux, without SystemD, can run on all of them and I can get back to making the cluster computing stuff “go”.
Note that the default passwd is “raspberrypi64” and that it doesn’t let mere mortals set passwords it doesn’t like (I.I. short or in dictionary or…) so you might need to “su root” to make that happen.
The dropdown menu shows distcc monitor installed and “which distcc” finds it already installed in /usr/bin/distcc so it is all set up for distributed compiles. ;-)
I’ve finished doing the basic “move in and make ready” on the RPiM3 Gentoo. (Add users, config hosts and fstab, etc.) I’m impressed. It will play video (with working audio) in the small size of panel as in a link in a posting, but has occasional ‘stutter’ of the image. Full size it has trouble computing enough and drops to 480 lines and still has a bit of image stutter. But for occasional small image video, it’s OK.
Moving windows and terminal windows is smooth and jitter free (something some other releases have had issues with…) so is likely using the GPU properly.
Gparted, Chromium, etc. all work fine (I’ve not tested all of Libreoffice, but it usually has no issues anyway.)
I’m presently making a backup copy of the chip, then will dup it onto another chip so I can try both PiM3 boxes with distcc as a “cluster of 2”. After that, I think I’m just going to be cranking out Gentoo boxes for a while. One board at a time, making them “go”.
I’m probably going to do the Odroid boards next. I’ve got a C1, C2, N2 and XU4. It looks like all of them have been done by others so ought to be fairly clear cut. The XU4 has weirdness, but the Gentoo HowTo is pretty detailed. I may also toss the Pine64 into the mixer too. It’s not my favorite board (has no heat sink so rapidly heat slows the clock way down… but maybe it’s time to order that heat sink…) but looks like lots of OSs port to it easily. I’m going to leave the RockPro64 as Slackware “for a while” just so that if I get in trouble converting the XU4 I still have a good fast Linux workstation that plays videos nicely ;-)
I think I’m going to emphasize Gentoo over the FreeBSD / NetBSD ports. I’ll still keep one or two of them in the pack as a “keep the skills up” and sometimes daily driver, but really, they have not been as robust in working (NetBSD and broken FireFox / no SQL) or in configuring (FreeBSD painfully non-configured Xorg). So I’m going to put more time into what IS working and hasn’t been that hard to “make go”.
I do need to install / test / port database for the MySQL / Python / Climate DB work I did some long time ago. IF that goes well, then I’m going to deprecate all the systems / chip images that is on.
For anyone with a Raspberry Pi M3, that “download and go” Gentoo image just works nicely right out of the chute AND all the choices the packager made have been nice ones. Even puts up some advisory panels on first boot for things like overscan, and tosses you right into the “demouser” demo account.
Interesting… There was “another browser” named Aurora installed. I just tried it. It is actually FireFox {and a 77.x release too, not the 52 level… so written in Rust).
It plays video full speed in the web page (560 or 640 bit wide) size and at 480p full screen without any video jitter. That makes the Pi M3 usable for general purpose INCLUDING video!
Oh, and the Gentoo come up in UK keyboard, but there’s a little box with “UK” in it in the upper right corner of the desktop that, if you click it, turns to US and you have a USA keyboard. Nice.
I’m REALLY liking the choices and overall build quality of this Gentoo!
Now I just have to show that I can “roll my own” of equal quality ;-)
EM – thanks for the report. If the PiM3 works out as a good daily system, seems like not that much of a need to get anything bigger for most people. The majority of the work I do these days just needs a competent browser, and occasional use of GIMP. For simulations etc. I can fire up the other boxes when needed.
@Simon:
The Pi M3 still has modest / slow disk speeds on USB. A copy of a 16 GB SD chip to my backup disk (that’s USB 3.0 capable) goes about 8 times faster on my boards with USB 3.0 ports.
The browser also starts to roll out to swap with multiple tabs open (I’m a tab hog and use lots of them) so my 2 GB memory boards handle that a little better
So all in all I like the RockPro64 and Odroid XU4 better. Then again, at about double the cost they ought to be better ;-)
The Pi M3 / Gentoo is certainly adequate. I don’t feel like I’m prevented from doing anything on them. Just once or twice something pauses for a second as the hard disk swap spins up (yes, I moved swap to real disk to reduce uSD card wear) or a page loads in the browser.
Oddly, the CPU is usually NOT fully loaded then. It’s the I/O subsystem that’s limiting then. I might just get a small Class 10 ultra SD card in a USB adapter and put swap on it to see what happens. I could see “burning” a $10 card every year or three as fast direct access swap…
Actually, thinking about it, I have a class 10 with Centos on it from some years back, but for the intel machine in the corner I’ve not used in 3 years… It’s full sized SD so doesn’t fit my SBCs (other than the old model B+ that already has a card…). It would be pretty easy to just store the image from it (or toss it…) and do the test….
Back in a bit? ;-)
FWIW, I found the Rock64 that is nominally about the same CPU as the Pi M3, to be noticeably faster / snappier, likely due to the memory and USB 3.0 stuff. And not that much more money.
I’d only chucked it in a box for a few years due to the “SystemD fstab Blank Screen Boot Bug”, where making an fstab entry of “noauto, 0 0” meant if the disk were absent your boot would just fail and give you a black screen causing it to seem like a dead board. Made me glade I’m an electronics pack rate when later I tried it with an entirely different OS and it just worked again. (Had the same issue with the Odroid C2 and another board. IIRC the XU4 was where I finally figured it out. I had an old non-SystemD OS chip and the upgrade to System D was done on a new chip. After fiddling some fstab settings, it “died”, yet the old chip worked.
Once the penny dropped on what was happening, I went back to the old boards, made sure all disks were plugged in OR the fstab entry was commented out, and suddenly they worked again.
That was the moment my dislike of SystemD turned into resolve to remove it from my shop as much as possible. A perfectly valid /etc/fstab entry ought not cause apparent complete failure to boot with no messages and no screen to log in and fix it.
Well, I was going to play with swap on SD card…
Well,
What I thought would be a quick little experiment with swap turned into a very long OMG! WTF? process…
I launched both Chromium and Firefox at the same time to generate a lot of swap activity. Firefox alone will cause about 300 MB of swap use with a 1/2 dozen typical tabs open. I opened a few more and got to about 450. Then Chromium had it rising past 600.
Then at about 671 MB of swap, something very important to the kernel or OS got swapped out and the system just froze.
Normally, you expect that kind of hang with NO swap and you run out of physical memory, OR with massive swap like 4 to 6 times physical memory. This was about 67% of physical memory. “Normal” allocation advice for swap is 1 or 2 x phys mem. That’s way low to be having a freeze.
So, OK. I repeated the experiment. With physical disk. With only ONE disk. With only ONE partition. With only one SD card in USB adaptor. With “sleep 3600; halt” to try to get it to auto shutdown if my terminal was hung but background processes were still running (it didn’t…)
So “something” in this fairly new 64 bit Gentoo is not staying memory resident when it ought. OK…
So I’m going to make a ‘special” swap of 600 MB and use only that. (Then we’ll find out if running out of swap also crashes things…)
In the mean time, this particular Gentoo is NOT for things that beat swap to death.
Per the Pi:
IIRC, it shares a channel for both the network device and the USB disks. That saturates fairly early on things like browsers that do both. Lots of red “D” disk wait symbols in “htop” display.
So the Pi M3 is suited to running ONE browser with not too many tabs open at once (stay below 600 MB swap…) and with a willingness for some things to load slowly as cached tabs (disk) and new tabs (network) are fighting over the one interface chip… Liveable, but not lovable.
Good for emergency use, when you don’t want to set up something else and this is already running, for kids, or for folks on a very tiny budget. Not so good for a College Kid looking to impress his or her friends… and spending $100 for a party that would instead get them a very nice 6 or 8 core 2 GHz SBC with 2 GB to 4 GB of memory instead.
I’m also going to need to watch big compile jobs as they can “saturate memory” pretty quickly if you set -j job number high.
With that, I’m done crashing the Pi for a while and I’m going back to getting Gentoo running on some other bit of hardware (where I’ll then find out if it too has the “crash on high swap” issue…)
FWIW, my guess is a device driver or other module and that a monolithic kernel build would prevent the swap crash. I had the problem on some other Linux about a decade ago… Every so often it shows up again.
My guess is also that the folks doing builds don’t have an experienced QA Manager abusing their systems to find where it breaks. At one time, I was the QA manager for a company making Linux compilers. My job as I saw it, was to stress test things until they broke. How else can you say if it is a normal failure, or not? Ideally the code would not break, but admonish me not to do stupid stuff. Far too often it silently failed.
So now you know why I do things like open TWO browsers at once with multiple tabs and video running in one of them. Because somewhere, someone, will do just that….
Well, as of now, I’m running Gentoo in a chroot on the XU4. I basically followed the directions here: https://wiki.gentoo.org/wiki/Odroid-XU4
With some minor convenience changes along the way. (Nothing structural, just things like mount point names). The current tarball to retrieve is:
stage3-armv7a_hardfp-20200509T210605Z.tar.xz
so in the wget you need to change to that (or whatever is current when you read this). I just went to the http site and read what was there:
http://distfiles.gentoo.org/releases/arm/autobuilds/current-stage3-armv7a_hardfp/
I didn’t create a clone of my existing XU4 OS chip, just backed it up to hard disk as a dd copy. Then I just put Gentoo in a partition on that chip. Also, I didn’t do long hand partition management, but used ‘gparted’ as it is just incredibly easier.
Finally, the fetch of the tarball, the unxz of it, and the untar were all done on the Raspberry Pi M3 as it was what was running at the time. I then booted the XU4 in Armbian / Devuan uplift.
To copy over the X11 directory, my mount point was /SD/Gentoo instead of /mnt so my command was: cp -L /etc/X11 /SD/Gentoo/etc
Similarly, the other commands mount point changed:
The “cp -L” says to follow symbolic links (i.e. don’t just do a link)
So now I’m chrooted into a running Gentoo on the XU4. Next up is all the various config and build stuff, but that’s likely to take a long time…
At the end of the “emerge -sync” is said to update the portage stuff, so I am:
Well that took a while, and compiled a LOT of stuff. It also threw a lot of “locale” warnings like:
I’m hopeful that:
A) As a warning it doesn’t matter. And
B) I can figure out how to fix it
C) It looks like it is defaulting to USA which is what I want anyway
This looks promising:
https://wiki.gentoo.org/wiki/Localization/Guide#Environment_variables_for_locales
emerge vim
has let me get the “vi” editor that my fingers know. Now I can stop using the horrid “nano” that uses CTL-{Letter} to do things… Yes, it is nice tor folks who do not know any Unix editor. OTOH, for anyone who’s fingers have been doing “vi” for 40 years, it’s a slothful thing that keeps erroring on the instinctive vi finger strokes…
But it is done now. Now I can try editing the “locale” stuff before finishing the install stuff per the script.
It would likely be better to have set the “locale” stuff before the portage update.
OK, I think I got “locale” fixed, but I’m not sure what all did it. On the thesis it was the LAST thing tried:
The /etc/locale.gen file had all options commented out. I “uncommented” the 2nd one, but that wasn’t enough. Uncommenting both US ones seems to have worked:
Prior to that, on advice of another plaintiff, I added LC_ALL=”” to /etc/env.d/02locale
IIRC I’d added the LANG= and LC_COLLATE= earlier. None of these seemed to fix it, but might have been necessary precursor steps.
I then did a “locale-gen”
And now it looks fixed. We’ll see if the warnings are gone on future emerge commands / builds.
This whole ‘locale’ stuff could really use a massive simplification. Just put your country code in one file and have everything else default (but over-rides manually available for someone who needs it…)
Oh Well.
The next “emerge –sync” seems superfluous to me, but I did it anyway. It appeared to sit there doing nothing after an opening stanza of stuff, and I thought to killing it… BUT, some sloth at the keyboard convinced me something must be happening. Some “Long Time Later” (after sushi dinner and 1/2 bottle of sake) is appears to have completed.
So, just realize that from this point on, there are long slow things that don’t talk to you a lot…
Well, the @world step runs for about 1/2 a day… It completed some time in the last 4 hours, but there were a couple of errors. At the very end is:
So, OK, I need to set a timezone and I need to try re-making python. Looks like I also need to dig through the log files and see what else might have failed.
During my spot checks (during dinner and into the evening up until about 5 hours ago when I hit the sack) it looked like everything was cooking along without failures. It would often saturate all 8 cores with gcc compile jobs. In the Gentoo wiki on the Odroid XU4 it says to put “-j 8” in the config (so run up to 8 jobs at once when compiling) yet says a couple of times to only put “-j 3” in the config file as more tends to “saturate memory”. My interpretation was that the “-j 3” was only when doing a kernel build, so I used “-j 8” , and everything seemed fine.
OK, what I’m going to do now is shut down the XU4, and make a backup copy of the chip. At this point I’ve got about 24 hours of serious compute crunching invested in it and it isn’t backed up. Then I’m going to contemplate log files over morning coffee, and after that proceed with the rest of the procedure. It is highly likely that I have a working system even if some of the @world packages failed to update when I did the “emerge –update –deep –newuse @world “.
The next three steps are all about cleaning out things no longer needed:
then adding the portage ‘gentoolkit’ to kernel generation. The revdep-rebuild does a reverse dependencies search and tries to figure out what bits depend on other bits to make sure it’s all there. Once all that is done, one is told to build a kernel. At that point I’ll likely change the config file to “-j 3” in keeping with my thesis about the conflicting directions. Worst case is that it takes a bit longer to build a kernel and I’ve got more cores left for me to use in the browser et. al. while the build runs ( I’ll change it back to -j 9 after the kernel build).
I can see where doing this on a Raspberry Pi would be a real challenge. The XU4 has 8 cores, with 1/2 of them being much faster, plus 2 GB of memory. My guess is that you would take about 4 to 10 times as long as a R. Pi M4, or a couple of days, just for the @world step. Cross compilation from a faster machine or using distcc across several would really matter a lot. (Or just download and go with the prebuilt image like I did ;-)
OK, enough of that. Time to shut down, start the backup, and go make coffee ;-)
Looks like how to set timezone et. al is here:
https://wiki.gentoo.org/wiki/System_time
and here:
https://wiki.gentoo.org/wiki/Handbook:AMD64/Installation/Base#Timezone
so I’m using UTC:
and it seems to have worked:
I may try a re-make of that Python package before doing the backup copy… but I’ll decide that AFTER that first cup-o-coffee ;-)
OK, coffee in hand, I did a simple check:
So I do have an installed python3 and the failed “emerge of dev-lang/python-3.8.5” ought not prevent the system from running. At most, the existing python3 might be back rev a little bit and that ought not cause any issues as minor revs are supposed to be compatible.
With that, time to start the backup, finish sucking on the coffee, and catch up on the morning news…
Well, the –depclean and related steps went very fast. I’ve now done all three of these:
This is the end of the install of gentoolkit and then running revdep-rebuild to assure all packages are consistent in their dependencies:
So looks like the next step is building a kernel…
I could make this bootable as-is with some potential for library conflicts in a manner similar to what was done for Funtoo. Just copy over the kernel modules and headers and use the existing kernel. I’m going to follow the script and build a new kernel anyway (despite the long build time probable and the more complex changes to boot.ini). It’s just more “correct” to have a kernel built for your system on your system with your system… and once done, that system image is no longer tied to any other build / system / libraries etc.
This is likely to take a while, so I may not say much on Gentoo until it is done. Today? Tomorrow? Who knows how long a kernel build takes on this machine… OTOH:
Unfortunately, they don’t say what “correctly mapped” means. I’m going to assume it means you know were /boot is located and boot.ini in it. But… some think time required…
This next bit looks like I need to check things are installed or install them (with emerge FOO) for a few things. Then the dance of the kernel sources begins…
Once that actually completes successfully, then the whole adjust boot.ini and reboot stuff happens. So, OK, maybe after a bit of a think, some checking installed tools exist, and breakfast. Doing an alien kernel build on coffee only is probably not a good idea ;-)
OK, not as long to next comment as I expected ;-)
First off, vi is giving me unreadable colored text. The dark blue of “comments” against black is impossible to read. So how to turn off that? Put “syntax off” in your .vimrc file.
https://www.cyberciti.biz/faq/turn-on-or-off-color-syntax-highlighting-in-vi-or-vim/
Second, why the ‘fstab is right’ and what IS right? Seems default /etc/fstab is just all example comments:
So what they are really saying is “Make an /etc/fstab that matches your system”…
OK, back to breakfast making ;-)
@EMSmith; I am sure glad it is you doing this and not me. Just reading all that gives me a headache. I think going out into the wood yard and splitting the winter supply of firewood sounds like more fun, 8-) A bit smoky here so we are, hanging out, inside with the Airwashers as much as possible.
Just listening to the MSM reporters complaining about the Doctors report on President Trump’s condition. I would guess that they were hoping to hear that he was dieing from the Covid. Sounds like these Doctors are following the “Approved” use of Resmedvier instead of something that might actually work, but, at least also using antibodies from blood donors.
They have you put /boot in a FAT partition in mmcblk1p0 but my Armbian just has it as part of the root ext4 partition. I’m going to try leaving it that way, avoiding the FAT boot style.
To that end, I’ve copied the Armbian /boot into the /SD/Gentoo /boot (that was empty) and will just build with that as the /boot target.
I’m guessing it will be fine. If not, I’ll have to move stuff around on the SD card to make room for a FAT /boot partition…
As usual “We’ll see”…
Not git or bc installed, so need to install them. I tossed in ‘htop’ too for monitoring activity on the chroot “machine” instead of just on the outer real machine:
So did:
odroidxu4 / # emerge dev-vcs/git {longish process with many compile steps happens...} odroidxu4 / # emerge htop {short fast install happens| odroidxu4 / # emerge sys-devel/bc {medium short install happens}The fetch of kernel sources took a little while (10? minutes?) but all in I/O waits on the transfer from what I saw (i.e. long distance internet slowest link)
I’m using /boot in the file system itself, so skip the following mount. (It will complicate the first multi-boot in that the Armbian boot.ini in partition 1 will need to be appropriate, then gets overlain with the one from the Gentoo partition, but when I copy this Gentoo partition to partition 1 of a new SD card, later, it will simplify that).
So I’ll do the cd, and the “git pull”, then it’s off to the actual genkernel. and most likely a long wait. Oh, and I need to drop to “-j 3” in the config file… /etc/portage/make.conf
Well, this is a disappointment… First off, “genkernel” is missing. Then, attempting to install it “has issues”:
In the original, the stuff after USE=” is in red” and so is “license changes”.
My guess is that now I get to go “dig here” on setting USE flags and how to deal with whatever that “license” issue might be. In any case, I still don’t have a “genkernel”:
IF this takes too long (ie. I’m too slow…) to figure out, I’ll likely just do it the Funtoo way. Copy over the kernel modules and firmware directories and boot off the Armbian kernel for a test run.
In any case, again, I’m off for a bit of a think and some reading.
Hopefully this running story of my experience with the Gentoo XU4 process will at least be amusing for others (and helpful notes for me some years on when I decide to try something like this again ;-)
OK, copied over /lib/modules and /lib/firmware from the Armbian partition to the Gentoo partition, changed boot.ini in partition one (Armbian slice) to point / at LABLE=SD_Gentoo and rebooted. Took about 2 minutes.
I now have a booted, running, native Gentoo on XU4. “Someday” I’ll need to sort out “genkernel” and finish building the kernel on Gentoo. But for now, I can bypass that hurdle and continue with installing things like a windowing system, XFCE or LXQt, Libreoffice, GIMP, etc.
At present, I’m logged in as root at a command line terminal. So it is working.
I think I’m going to make another backup copy and take a news break… then learn more about USE flags, genkernel, et.al.
Well… went to install a package and found that networking isn’t working. OK…
So I’m back running Gentoo in a chroot as I add things. So now 2 things on the todo list. Fix networking (that will be easier once install the networking tools bundle) and X11 / windows can then be installed; plus that kernel build “someday”.
I’m continuing to follow (more or less ;-) the Gentoo guide at:
https://wiki.gentoo.org/wiki/Odroid-XU4
Note that “layman” is an overlay manager for the Gentoo build system. The argument is an CAPITAL S for “sync all”, if you use a lower case “s” it complains that it wanted an argument (of ALL)…
But the next line puts me back in Portage Odd Stuff land with “masking” issues (whatever they are) to sort out:
The page then goes on to say how to make networking go in an OpenRC world:
so I’ll boot up again and see if that worked…
I already have an /etc/resolv.conf set, so don’t need to re-do that step.
And I’ve got the “Q” Quiet version of the XU4 without a fan, just a really big heat sink, so I can ignore the fan control stuff.
Which puts me at the end of the script.
IFF the networking is now working, then “when” I get the video driver masking sorted, I can then do “all the usual” as it says:
and maybe, just maybe, get more than a CLI interface…
Hooray! Hoorah! I have network working in native Gentoo boot!
That puts me at the end of the linked page guide. So I’m basically done with the cook book for now.
I have a booted, working, up to date Gentoo running on the Odroid XU4.
It does not yet have a windowing system installed. It needs a native kernel build. I need to load a lot of added software. Then, the biggie, I need to learn about their package manager and build system. Masking, USE flags, and more.
Not ready to be my daily driver, but far enough along to join a distcc cluster for builds, and can add software in native boot (i.e. not via chroot).
I think that’s pretty good progress. Time for some reading manual pages (and a nap 9 -)
A note on USE flags.
I think I’ve gained clue on them. They are a nested (almost recursive…) layering of what to leave out or put in your OS build. So, for example, if you need Python, you can specify just WHICH level of Python needs to have support included. So if you are using old 2.7 you need not build support for 3.5 (or others…). Similarly, you can include only drivers for YOUR video card. Or different graphics “kits”.
They are specified in a default “profile” (that I’ve not yet found for the ARM boards so who knows) and several pages described how to shut off the default profile anyway. It looks like just a generic description of what works on a class of machine, and with ARM being highly variable, probably of less use.
Then there is the portage config file that has a USE line / override. Naming a flag includes it while the name with a minus sign in front removes it. Then there’s a directory where you can list, per package, things to include or leave out. Finally, on the actual build command (emerge FOO) you can over ride all of that with another “USE=” statement.
I suppose all this is useful for those times where you have a default profile, but need to change some stuff (while not wanting to deal with all of it) but then find one package that wants, say, Ruby24 support and you built with Ruby25 support…. HOWEVER, it’s a bit of a PITA when you walk into it cold with all that stuff in layers… More on that here:
https://wiki.gentoo.org/wiki/Handbook:AMD64/Working/USE
https://wiki.gentoo.org/wiki/Profile_(Portage)
So, OK, I looked at the USE settings on the RPi M3 “include all the goodies” build, and it has a long list.
Then I looked at the USE settings on the build for the XU4. It says in the description “I used minimal USE flags” or some such. When I first read that I thought “Oh, good, fewer to figure out”. Now I realise that means “I left out lots of stuff to make the build faster and smaller”…
Then I compared the two and tried to come up with what I think is a good list for a full featured XU4 build. Realise that for many of these I’m guessing. Mostly I just added in what the RPi had if the XU4 didn’t have it. So turned “RUBY_TARGETS=”ruby25 ruby26″” into RUBY_TARGETS=”ruby24 ruby25 ruby26 ruby27″ since the RPi had RUBY_TARGETS=”ruby24 ruby25 ruby27″. I figured the ruby24 must have been for some package on the Pi that needed it, and the ruby27 was likely a new version that came out after the original XU4 build was made. It will make a slower build process (building more stuff) and a bigger OS (more stuff built) but the XU4 is a lot bigger than a Pi anyway ;-)
http://phonedb.net/index.php?m=processor&id=483&d=detailed_specs
The trickiest bit is likely the CPU spec parts. The XU4 had:
CPU_FLAGS_ARM=”edsp thumb thumb2 v4 v5 v6 v7 vfp”
while the RPi has:
CPU_FLAGS_ARM=”edsp thumb thumb2 v4 v5 v6 v7 v8 vfp vfp-d32 vfpv3 vfpv4″
The XU4 build is based off of he generic build, so limits to generic hardware.
https://packages.gentoo.org/useflags/cpu_flags_arm_edsp
thumb and thumb2 are compact instructions used sometimes. v4, v5, v… are different era of instruction sets and ARM chips typically support older versions (often of shorter bit width so v7 is 32 and v8 is 64 bit while older ones can be 16 bit). I removed the v8 from the list as the XU4 is a 32 bit machine. Then, we have the vfp vpt-d32 vfpv4. Also, I’m interested in using video cores / NEON instructions for parallel math, so want NEON instruction set support included.
https://forum.beyond3d.com/threads/arm-vfp-vs-neon.45588/
SO I combined AND expanded to:
CPU_FLAGS_ARM=”edsp neon thumb thumb2 v4 v5 v6 v7 vfp vfpv3 vfpv4″
I left out the “vfp-d32” as that says your Floating Point Unit as 32 x 64 bit registers and I think that the V7 likely does not. But that’s a guess.
What I think that says is to build support for “the usual” historical instruction sets and thumb instructions, AND include graphics instruction sets for 4 eras of video cores (figuring the XU4 chip is pretty new it ought to have the newer ones, but maybe being v7 it doesn’t and I’ll need to trim things back one at a time until it works… those vfp vfpv3 etc items) Yes, I ought to look it up instead of “build and break maybe” but where’s the fun in that? Also support the NEON instructions.
Also on Input Devices, I just tossed in a few more. “synaptics” is for the mousepad driver.
INPUT_DEVICES=”evdev libinput synaptics”
Supposedly synaptics over rides evdev if both are present, but the RPi had both. The XU4 had libinput and I figured that’s likely some minimal bit. I decided to just build all 3 (on the theory that more is better and stuff can choose which one to use…)
https://packages.gentoo.org/useflags/input_devices_synaptics
https://wiki.gentoo.org/wiki/Synaptics
https://xorg-team.pages.debian.net/xorg/howto/configure-input.html
The only other odd bit is QEMU support. The RPi was build with support for QEMU virtual machines. I think it a bit daft to run a VM on a Pi, but hey, I like doing daft things to abuse hardware sometimes…. So I added those lines to the XU4 build, BUT the RPi is 64 bit and the XU4 is 32 bit so some guesses on changes were made:
QEMU_SOFTMMU_TARGETS=”aarch64 arm i386 x86_64″ QEMU_USER_TARGETS-“arm”
What I think that says is to be prepared to run VMs using 64 bit arm, and 32 or 64 bit Intel instruction sets, and run them on a 32 bit ARM target. Compare the RPi entry:
QEMU_SOFTMMU_TARGETS=”aarch64 arm i386 x86_64″ QEMU_USER_TARGETS=”aarch64″
So my guess is that I need “arm” where it has “aarch64” (and again, yes, I ought to look it up and make sure it isn’t aarmv7 or whatever…)
These are just one very very very long line of text, and rather than crap up a comment with 90 lines of line wrap, I’m going to include the three versions of
Now, one other minor problem. I copied these two configs to a work directory where I could compare them and make my own version… and now, the next morning, forgot where in the name space they live… Sigh. Didn’t write that in my notes. So, OK, more coffee and try to recreate what I knew last night… For now, here’s the three versions each on one VERY long line extending way off to the right ;-)
The XU4 with generic build:
The PiM3 built with all the goodies:
The XU4 proposed all the goodies version:
So once I figure out where I found these, I can test my proposed changes. ;-)
UPDATE:
Ah, the miracle of coffee and a warmer afternoon body temperature… I didn’t get these from a file, I got them via a query command:
Which is giving me the net-net of the whole Profile, /etc/portage/make.config, etc. etc. stack of USE flags.
So what I need to do is:
a) Accept that the “profile” just is, as there are several admonishments about them being a pain.
b) Look to my build make.conf as a place to overlay the desired changes to global USE flags.
c) Perhaps, add some per-package USE changes if needed.
OR…
I can in fact take on the pain and suffering of making a new Profile (once I figure out their add bits) and customise that.
Well this is really useful. Just wish I’d have found it days ago… or maybe weeks…
Ok, I’ve gone through all those USE options and identified the ones that ought to be added. Many of them are various codecs or support for CD / DVD formats reading / writing. As I don’t have such a device on the UX4, I’ll likely leave those out on my first cut.
The major goal is to get X windows working. For that I need to add “X” and likely “GTK” + “XCB”; plus some of the sound support (“alsa”, “pulseaudio”, , some codex types) and maybe some graphics format support (“PNG”, “TIFF”, “SVG”).
I need to either choose to put them in a new profile (probably not at first) or in the make.conf (highly likely until stable). Some might be best added on a “per package” basis, but I don’t have enough information to know which ones and what packages just yet. So maybe for later.
“Masking” is where a profile says “Don’t use Foo.1.2.3.5” because it is either broken, ceased support years ago and is highly risky, or some other reason. I’m going to try initially to just accept whatever default masking exists and only dive into it if something I really want blows up.
Initially also I’ll be leaving the various language versions as definded in the existing XU4 bld.
For example, the Pi has Python version 2.7, 3.6, and 3.7 selected while the current XU4 build has Python 2.7 and 3.7 only. IFF something breaks and demands Python 3.6; well, I know how to add it. Same thing for Ruby 24, 25, and 27 vs just 25 and 26.
I’m also going to leave out some things I doubt I’ll need. Like QEMU .
So now I’m off to take a crack at making the USE command in the make.profile and trying another build. But while this box (The Pi M3 Gentoo) shuts down and the XU4 boots up, I think I’ll make a cup of coffee… I think I might need it…
After a few conflicts centered on Python levels (I think) I cut back the USE flags a lot, to mostly just X stuff and some image formats. Now it seems to be building OK with 82 new packages to install.
With luck, this will get X working and I can get a set of windows going. That’s the big hurdle now. Once that’s done, the machine is a whole lot more comfortable to build on. Errors more easily captured. Build in one, monitor logs in another, etc.
So now I wait… or check back in the morning ;-)
Pingback: SBC OS Choices & Opinions | Musings from the Chiefio