Alpine Linux – Small, Fast

I was making a matrix of releases and features, one of which is “MUSL” (or alternatively uClibc) build for small fast secure operation. Usually those two alternative libraries are used for things like embedded systems or routers. Places where size matters rather a lot since they are often very starved for hardware. Many still use 16 bit chips and memory measured in megs.

There are a few desktop systems built with them too, but often that is “experimental”. One is Sabotage, that is a port of Linux From Scratch (LFS) using MUSL and with an abhorrence of the HFS (file system name space standard). As I’m rather fond of already knowing where things are located, moving them all around again on theoretical grounds (good though they may be) doesn’t excite me much… I may yet build Sabotage just to experience it, but learning yet another package manager and having yet another place where “everything you know is wrong” on name space is low on my ‘Oh Boy!’ list…

While digging around there, I found that Alpine Linux is already built on MUSL. Not only that, it has a hardened kernel build. They have a default “run from memory” mode and use OpenRC, not systemd. They have a port that runs on the Raspberry Pi. Since all of those are things I wanted, it is an obvious place to try.

http://alpinelinux.org/

From their “about” tab:

About

Alpine Linux is an independent, non-commercial, general purpose Linux distribution designed for power users who appreciate security, simplicity and resource efficiency.

Small

Alpine Linux is built around musl libc and busybox. This makes it smaller and more resource efficient than traditional GNU/Linux distributions. A container requires no more than 8 MB and a minimal installation to disk requires around 130 MB of storage. Not only do you get a fully-fledged Linux environment but a large selection of packages from the repository.

Binary packages are thinned out and split, giving you even more control over what you install, which in turn keeps your environment as small and efficient as possible.

Simple

Alpine Linux is a very simple distribution that will try to stay out of your way. It uses its own package manager called apk, the OpenRC init system, script driven set-ups and that’s it! This provides you with a simple, crystal-clear Linux environment without all the noise. You can then add on top of that just the packages you need for your project, so whether it’s building a home PVR, or an iSCSI storage controller, a wafer-thin mail server container, or a rock-solid embedded switch, nothing else will get in the way.

Secure

Alpine Linux was designed with security in mind. The kernel is patched with grsecurity/PaX out of the box, and all userland binaries are compiled as Position Independent Executables (PIE) with stack smashing protection. These proactive security features prevent exploitation of entire classes of zero-day and other vulnerabilities.

Essentially, the first cluster of my goals for a linux. It would be far far easier to start from their base rather than redo all of that myself, and likely me doing it less well. They state that historically they had begun as a Gentoo fork (or copy), which was also a strong candidate for me too. All in all, highly promising.

So I installed it, following the directions here:

http://wiki.alpinelinux.org/wiki/Raspberry_Pi

The install is very fast and very easy. Essentially download a tarball of about 79 MB, extract it onto a Fat32 formatted SD card, stick that in the Pi and boot.

That’s it.

No, you don’t need to pay attention to Pi or Pi Model 2, it figures it out… Well, you need to use the right SD or micro-SD for your box…

I installed it on a crappy PNY 8 MB class 4 card (that failed to boot Raspbian…) since that was what I had empty. It booted and ran fine and fast.

There are some config steps you go through, listed on their page, but nothing particularly hard. The major thing was just how startlingly fast it was at booting, at handling packages, et everything. All this with a 200 ish MB memory footprint including running from a RAM disk!

If nothing else, it is a stellar proof of concept for a small fast secure distribution based on MUSL and with a hardened kernel.

The Bad

It has Yet Another Package Manager. Called “apk”. it is sort of like pacman and sort of like apt-get and sort of like… but seems to work well. Doing an “apk add FOO” on a few things was OK and very very fast. But really, we can’t standardize on one package manager name with optional flags?

The biggest issue for me was that the directions for Setting Up X and getting xfce desktop running didn’t work. Typing “startx” gave me a black screen and a locked up system (well, it might have been running, but with monitor, keyboard, and mouse out of action…) For further debugging, I’ll need to set up a second box so I can rsh into it to kill the X process when it locks the screen… Sigh.

OTOH, while I rate my desire to do debugging of X-windows problems slightly below tooth extractions, root canals, and playing on fire ant hills, it is a lot less work than starting a port from near scratch to end up debugging your own X-windows problems… At least some folks have gotten it to to work already on Alpine.

I found one chat log from late 2015 where one of the developers said, in response to someone relating troubles, “getting a desktop going in Alpine is a labor of love”…

In Conclusion

As it stands now, I’m very impressed with Alpine for its stated goal of headless and embedded systems. I will most likely covert my Pi B+ to it as my DNS and misc. server since it always runs headless anyway.

Over time, I’m slowly going to whack on it when there’s nothing immediate on the plate, to see if I can find the formula to make lxde or xfce or whatever desktop run.

However, at the present moment, it is too “user hostile” for setting up a desktop for use by the average Joe or Jane. Since one of my goals is a completely “cook book” install with as much as possible built locally from source code if desired, my present first target is unlikely to be Alpine. (But who knows, I might discover some simple PIBKAC error in the install and be all over it tomorrow…)

As of now, the most likely candidates are Slackware and LFS. LFS has a small lead since it has a CLFS version based on MUSL already running. Slackware comes with “distcc” already built and installed (!) so clearly has a head start in the area of “built to build”, but I’m still working on their build packages system (slackbuilds). It does a lot of ‘from source’ stuff, but it is all user driven with hand reconciliation of dependencies… Not a NOOBS kind of thing. LFS is “follow the cookbook” clean, though with fewer options and packages. Decisions decisions… More on that in another post later.

OK, enough on that. Alpine is impressive kit inside their stated domain, “needs work” on the desktop front. I’m gonna whack on it at medium low priority over time. IF it had a larger user base, it would be a simple web search, but as a low usage distro, especially on the Pi, that didn’t turn over much. So up to me, if slowly. Anyone else wants to “run out ahead”, feel free. I’m going to try a LFS build next and see how many months it takes ;-)

Subscribe to feed

Posted in Tech Bits | Tagged , , , | 11 Comments

Natural Gas, Coal, and Cheap Plentiful Energy

There’s folks in Europe trying desperately to sell the idea that we are “RUNNING OUT!!!” (cue village idiot running around with big frizzy hair and a BBQ lighter stuck in it, rapidly pulling the lighter trigger…)

Well, we are NOT running out. In fact, the “trouble with coal” is just that we have too damn much energy supply. Hundreds of years of it. If and when price gets too high (for any of the competing fuels of coal, gas, oil), we break out the tools to make more available, then prices become too low and shut down exploration. In an ideal world, that would stabilize at a constant rate of exploration and production matched to the rate of decline of existing fields, but discovery (of fields or of methods) is not a continuous process, it is a point process, so the system ‘rings”…

This has been known almost forever. The Railroad Commission of Texas was set up to attempt to stabilize the boom / bust nature of oil “way back when”. It was OPEC before there was an OPEC and in some ways served as the model for OPEC when oil went more global.

The Railroad Commission of Texas (RRC; also sometimes called the Texas Railroad Commission, TRC) is the state agency that regulates the oil and gas industry, gas utilities, pipeline safety, safety in the liquefied petroleum gas (LPG) industry, and surface coal and uranium mining. Despite its name, it no longer regulates railroads.

Established by the Texas Legislature in 1891, it is the state’s oldest regulatory agency and began as part of the Efficiency Movement of the Progressive Era. From the 1930s to the 1960s it largely set world oil prices, but was displaced by OPEC (Organization of Petroleum Exporting Countries) after 1973. In 1984, the federal government took over transportation regulation for railroads, trucking and buses, but the Railroad Commission kept its name. With an annual budget of $79 million, it now focuses entirely on oil, gas, mining, propane, and pipelines, setting allocations for production each month.

Then along came the shale oil and natural gas booms… outside of The Railroad Commission of Texas and outside of OPEC.

Now natural gas is incredibly cheap. How cheap? Well, first some “units stuff”. Coal and gas here in the USA (where it is cheapest much of the time) is measured in “Million BTU” chunks. It is a convenient size, so all you folks afflicted with “One size never fits” SI units, get over it. How big is a MMBtu?

http://www.convert-me.com/en/convert/energy/mymmbtu.html

will let you turn it into whatever you want. It is 10 therms (what my gas bill from PG&E prices) or 1,055,000 kJoule (so a kJoule is almost the same as a MMBTU… It is also about 293 kiloWatt-hours and about 36 kg of coal (though various types of coal move that around a lot). For natural gas, it is roughly 1000 cubic feet ( 970 of ‘standard’ cubic feet). In short, it’s a lot of energy.

How much does it cost right now?

http://www.eia.gov/dnav/ng/hist/rngwhhdd.htm

has natural gas prices, both historical and present. There is a spike about 2003 to near $18 / MMBtu, then a broader topping peak in 2006 at $15, convincing folks it was worth it to “do something”, then all over the continent folks started developing the harder to produce gas (via things like fracking and coal seam gas and more). The price now? The 5 prices are the days of the week for each week:

Henry Hub Natural Gas Spot Price (Dollars per Million Btu)
Week Of 	                Mon  	Tue  	Wed  	Thu  	Fri 
  2016 Apr-18 to Apr-22 	1.76 	1.94 	1.94 	1.96 	1.92
  2016 Apr-25 to Apr-29 	1.97 	1.97 	1.88 	1.88 	1.89
  2016 May- 2 to May- 6 	1.91 	1.94 	2.03 	2.05 	1.86
  2016 May- 9 to May-13 	2.01 	2.01 	2.01 	2.01 	1.96
  2016 May-16 to May-20 	1.91 	

So bouncing around near $2 / MMBtu. How big is a gallon (US) of gasoline? Well, the modern stuff is diluted with alcohol, so not as valuable as it once was, but roughly:

https://en.wikipedia.org/wiki/Gasoline_gallon_equivalent

Fuel: liquid, US gallons 	GGE 	GGE % 	BTU/gal 	kWh/gal 	HP-hr/gal 	Cal/litre
Gasoline (base)[3] 	        1.0000 	100.0% 	114,000 	33.41 	        44.79 	        7594.0
Gasoline (conv., summer)[3] 	0.9960 	100.4% 	114,500 	33.56 	        44.99 	        7624.5
Gasoline (conv., winter)[3] 	1.0130 	98.72% 	112,500 	32.97 	        44.20 	        7496.5
Gasoline (ref. gasoline, E10 	1.0190 	98.14% 	111,836 	32.78 	        43.94 	        7452.4
[...]
Gasoline (regular unleaded)[5] 	1.0000 	100.0% 	114,100 	33.44 	        44.83 	        7594.0

About 0.11 MMBtu. Since gasoline is presently running about $2.50 / gallon, you can see that it is about 10 x as expensive as natural gas in bulk industrial quantities.

How about coal? Well, coal is driven by price competition against the alternatives, and now by the heavy hand of government destruction via regulation, and what is that price now?

http://www.eia.gov/coal/markets/

Average weekly coal commodity spot prices
dollars per mmbtu 	Week 	Week ago
                        04/15   04/22	04/29	05/06   05/13 	change
Central Appalachia
12,500 Btu, 1.2 SO2 	$1.69 	$1.69 	$1.62 	$1.62 	$1.62 	$0.00
Northern Appalachia
13,000 Btu,<3.0 SO2 	$1.79 	$1.79 	$1.68 	$1.68 	$1.68 	$0.00
Illinois Basin
11,800 Btu, 5.0 SO2 	$1.34 	$1.34 	$1.34 	$1.34 	$1.34 	$0.00
Powder River Basin
8,800 Btu, 0.8 SO2 	$0.53 	$0.53 	$0.53 	$0.53 	$0.53 	$0.00
Uinta Basin
11,700 Btu, 0.8 SO2 	$1.62 	$1.62 	$1.62 	$1.62 	$1.62 	$0.00 

For high sulphur, low density, Powder River coal, it is 53 ¢ / MMBtu, or roughly a nickle / gallon of gasoline equivalent. Even low sulphur coal is priced 30 ¢ / MMBtu under natural gas due to the hand of regulations. Even there, it is 17 ¢ / Gallon Of Gasoline equivalent.

In any kind of sane world we would be converting natural gas and coal to gasoline. The technology for both is well proven and used in South Africa today. (At one time Mobile Oil in New Zealand converted their natural gas to gasoline, but I believe that project was abandoned when OPEC crashed prices to drive out the competition).

There is exactly zero non-political reason for anyone anywhere on the planet to be paying insane prices for energy, as is happening in the UK and Europe.

Now one could speculate as to why prices are so high… Ever since the start of the Texas Railroad Commission and later OPEC, the single goal has been to drive prices higher and stabilize them there. The families made rich off of oil and coal have not forgotten, nor gone away; yet now they support the “Green Movement” heavily. Think just a moment… Could it possibly be that they have simply found another vehicle to drive consumer payments “way high” for something that is literally “cheap as dirt”? Hmmm?

The raw material from which most of USA electricity is made runs about 190 / 293 ¢/kW-hr or well under a penny. Yet PG&E with summer time of day pricing wants $1 each for it in the Central Valley of California (but “only” 19 ¢ headed for 35 ¢ each other times…)

The simple fact is that we have no shortage of energy supplies at dirt cheap prices.

What we do have is a manipulated sewer of crony capitalists, government graft, organized monopoly cartels, and a long history of “screw the public”. The Green Blob and “Global Warming” scare as just the latest embodiment in a very old game….

Subscribe to feed

Posted in Economics - Trading - and Money, Energy, World Economics | Tagged , , , , | 31 Comments

Trump’s Problem With Women

I’ve lost the pointer to where I found the link that sent me here…

http://www.anncoulter.com/columns/2016-05-18.html

TRUMP’S PROBLEM WITH WOMEN
May 18, 2016

The New York Times’ front-page article last Saturday on Donald J. Trump’s dealings with women forced me into a weekend of self-examination. As much as I support Trump, this isn’t a cult of personality. He’s not Mao, Kim Jong-un or L. Ron Hubbard. We can like our candidates, but still acknowledge their flaws. No one’s perfect.

I admit there are some things about Trump that give me pause. I’m sure these will come out eventually, so I’m just going to list them.

First — and this is corroborated by five contemporaneous witnesses — in 1978, Trump violently raped Juanita Broaddrick in a Little Rock, Arkansas, hotel room, then, as he was leaving, looked at her bloody lip and said, “Better put some ice on that” — oh wait, I’m terribly sorry. Did I say Trump? I didn’t mean Trump, I meant Bill Clinton.

Hang on — here we go! Knowing full well about Bill Clinton’s proclivity to sexually assault women, about three weeks after that rape, Trump cornered Broaddrick at a party and said, pointedly, “I just want you to know how much Bill and I appreciate the things you do for him. Do you understand? Everything you do.”

No! My mistake! That wasn’t Trump either. That was Hillary Clinton. … But this next one I’m sure was Trump.
[…]

Hit the link to read it all… well worth it to “remind” all the FUBARs done by Billy Boy while being Molester In Chief…

Then ponder him hanging around the White House with nothing to do but “help” the interns…

Madam Hillary has BIG problems in Bill, and saying she would delegate some of her responsibilities to an unelected / retired old goat does not improve them…

Subscribe to feed

Posted in Political Current Events | Tagged , , , , , | 5 Comments

End Of Germany?

“Demographics is destiny” a fundamental truth of Economics and Population Dynamics.

So over on Tallblokes, Gail Combs posted an interesting link. The site is, er, a bit inflammatory in many ways, so I can’t endorse it on the basis of just one or two videos. Yet the demographics in the video are accurate.

The basic thesis is that the influx of a huge load of young men from non-German countries will demographically swamp the roughly equal sized young german male demographic. The implication is “the end of Germany”, as a country and a culture and a genetic group.

Is there reality in it?

IF present trends continue, yes. But “all things being equal” is rarely kept equal…

On that site, I ran into another video. It is even more disturbing. Both for the ideas it puts forth, and for the fact that they cite a respected body of literature in their conclusions. Yes, the cite of culture as derivative from sexually repressed cultures is from old grey beards like Frued and J.D. Unwin (1895-1936), so could be criticized as musty and out of date (and perhaps a cherry pick to match the video thesis…)

Yet it does cause a fair amount of thought, matches what I know of the biological basis of sexual behaviours and “in group” vs “out group” preferences, and echos the thesis about genes driving one type of person to Progressive and another to Conservative.

I can’t particularly endorse it, OTOH, once past the gratuitous inflammatory hype sellers puff, I can’t point at any given thing in it and say “Well, that’s quite wrong”. Mostly it makes me think. I like things that make me think. ( I don’t have to agree with the content or conclusions to like thinking about something… )

At any rate, hopefully it will lead you to think a bit too. Like it or not ;-)

Subscribe to feed

Posted in Human Interest, Economics - Trading - and Money, Political Current Events, Religion, Biology Biochem | Tagged , , , , , | 28 Comments

Slackware on Pi Notes

To install Slackware on the Raspberry Pi, I was greatly helped by this page:

http://rpi2.fatdog.eu/index.php?p=home

It is essentially a well written and easy to follow cook book for putting Slackware on the Raspberry Pi Model 2.

I have but one complaint: Despite it telling me a half dozen times “DO NOT REBOOT!!” in red, even, after 2.5 hour of code download and another 2.5 hour of install script running, when the script said “reboot now” I chose to reboot. Sigh. squirrel!!

That put me off in “recover a botched install” land. Not as hard as it sounds. I copied the whole system ( from / directory on the chip on down) to my USB drive with a simple tar command: tar cvf /SaveDisk/Slack.tar or something similar) and then just reflashed the SD card with the right boot images and ran through enough of the install to get things back like they needed to be, then restored the Slack.tar blob to the chip and did NOT reboot…

OK, ok… they warned me. Many times. It isn’t THEIR script / install process that offers the tempting reboot; so they can’t fix it, etc. etc. blah blah blah… My bad, I got it…

It is also the case that, as usual, they have you using low level drive formatting commands for the SD card. As usual too, I would normally do it using gparted under Arch (or Fedora or Debian or…) and be done with it, working around that part of the process. In this case, I actually used their command. It gives one of those ncurses like ‘almost graphical’ panels made of text fields… Not too hard to do, but really… welcome to 1980…

http://rpi2.fatdog.eu/index.php?p=infodrives

cfdisk /dev/mmcblk0
[…]
The vfat (FAT32) partition ‘mmcblk0p1’ is fine as it is. You’re going to use that as your boot partition but you’re also going to need additional partitions in order to house your Linux system. So, at this point you should create a swap partition and a root partition.

Create a swap partition like this:

• Move the highlight down to the Free Space, using the cursor keys on your keyboard.
• select [ New ] at the bottom, and then press the enter key.
• Select [ Primary ] and press the enter key.
• When asked to specify ‘Size (in MB):’ type 256 and press the enter key. You can enter your own size if you prefer.
• Select [ Beginning ] and press the enter key.

Your new partition has been created. Now you need to tell cfdisk that this partition is going to be used as a swap file. At the bottom of the screen move the highlight to [ Type ] and make sure that ‘mmcblk0p2’ (the 256Mb swap partition) is the one still highlighted. Then press the enter key.

I understand the desire / need to be entirely self contained and self hosted, but really, other than your first bootstrap machine ever, chuck the card in a box with gparted and be done in 60 seconds…

They give a nice top level overview of the process on the entry panel to the build:

http://rpi2.fatdog.eu/index.php?p=installer

Slackware ARM Installation Guide

Welcome to the SARPi2 installer guide. This guide is a step-by-step, end-to-end, HOW TO install Slackware ARM current on a Raspberry Pi 2 model B. If you have no need of this guide, the SARPi2 installer and system packages are available from the Downloads page.

Some sections of this installer guide are marked with an [option] which means you can choose to follow them, or not, depending on how you intend to carry out your Slackware ARM installation. Each section is linked to the next consecutive section at the bottom of the page to hopefully give this guide a little more user-friendliness and continuity.

If you’re using this website for the first time, it’s advisable to follow the links below starting from the top. When you’re ready…
Prerequisites

• Preparation before you begin
• Configure a microSD card
• Download Slackware ARM source media [option]
Slackware ARM Installation

• 01. Notes for Pre-installation
• 02. Configuring the system for installation
• 03. Setting up the NIC for (remote) installation [option]
• 04. Creating partitions on available drives
• 05. Mounting a USB memory stick [option]
• 06. Running Slackware ARM setup
• 07. Selecting Slackware ARM source media
• 08. Final steps of installation
• 09. Completing the install process
• 10. Configuring the boot partition
Keeping Updated

• Keeping Slackware ARM updated with slackpkg
• Upgrade kernel, modules, and firmware with rpi-update
• Optimise Your System Configuration [option]

Each of the bullet items a link to that step set of directions.

I used a USB disk, not a stick, and it worked fine. My only “issue” there is that the directory you give is a bit less than clear. I had it mounted as /Slack and filled with the download / unpacked source and build bundle. You do NOT say “install from mounted directory ‘/Slack'”… It is /Slack/slackware as they use a directory one level lower than some of the stuff… OK, got it… after a failure to install with /{mountpoint} …

It took me about 2 1/2 hours to download everything, and another 2 1/2 hours for the actual install process (that was mostly spent watching TV as it just runs…). I chose to install absolutely everything, so YMMV if you accept less. (In particular, the fonts for internationalization they say to skip are big and mostly useless to me.)

Another mistake I made was to say “Sure, install dnsmasq”. This fights with BIND for port 53. You are not expected to run them both at the same time. My bad. While I do run dnsmasq on another box under raspbian, I don’t really need it here. BIND is fine. Either that, or realize you are responsible for changing the config to only run ONE DNS resolver using port 53…

The other minor goblin I ran into was that time was being a pill. It often is. IFF you are too far off from the time reported by a time server, ntpd will refuse to ‘drift’ you to that time. It decides that your clock could not have just drifted 45 years during a shutdown… This fails to service a Raspberry Pi well, as it has no hardware clock. I knocked together some mini-scripts that stuff the present time into a file at shutdown and read it back in to set the clock at boot up and many ‘need to set the clock by hand’ issues ‘went away’. I’ll post the exact details when I get the Slackware chip back in and booted again. IIRC, in the rc.d for ntpd I added a line to set the date, and in a new /etc/local_shutdown script added a command to write the current clock setting out to that same file. Each command in /usr/local/sbin and a ‘one liner’ like date -s ‘cat /etc/SaveTime’ or some such.

After that, all was good.

General impressions?

I like the BSD style rc.d init control scripts. Took me a couple of minutes to remember to forget the init.d SystemV stuff and revert to BSD ;-) So not only does Slackware have the good taste to call systemd crap, they didn’t even get hooked withy the “System V Consider it Standard” crap either. Nice ;-)

The original rc.d init system is clean, simple, and easy to modify, so in about 5 minutes I had my clock workaround in place and tested. I have no clue how to fix it under the systemd overlord… I’m really liking the BSD init glow ;-)

It seems fast. No, I didn’t do any timing yet. It is just that you notice when things are ready and you are not yet… that “but wait, I reached for the coffee cup but have to STOP before the sip? That’s new…” thing.

Konqueror was a dream at reading pages and making comments. Due in part to the lack of constant live spell checking of every word in a loop. You select when to spell check and it then does a whole doc check presenting you with a list of dictionary words for each thing where it thinks there is an error. Partly just for who knows what reason. CPU use down in the 7% range most of the time. Rarely over 17%. Snappy and with no ‘type ahead’ issues on comments. My one complaint here is that attempting to write an article ended badly. I lost the “save” button. Maybe it is a ‘feature’ and I had some toggle turned (it also jumps to a new window on some mouse things… another miss-feature gaining wider use…) but “suddenly” and against my will I could not save the article. I’ll go back to it later and explore a bit more. Figure out if it is a PIBKAC problem or not. (Problem Is Between Keyboard and Chair)

That there is exactly ONE browser and not a one I regularly use is an issue for me. Now I’m either off in the “install another ARM built browser on a strange system” land, or … (Not many folks expect ARM / Pi Model 2 as their typical download / install target, so you wade through all the PC M.Soft and Mac stuff to find it.. then get the x86 AMD64 Linux options out of the way then… ) So unlikely to become my “daily driver” unless and until that is fixed, somehow. No, not “hard”, just time that I don’t want to spend at the moment. The time budget drives all…

FWIW, the Original Article Start

Just as a FWIW thing: I’d managed a ‘copy paste save’ of the original article I was writing on Slackware on Slackware.. Here it is, just for grins:

BEGIN QUOTE:

I’d run into some difficulties getting X-windows installed on Gentoo, so thought I’d just “limber up” with a vanilla install of something else that didn’t take so much detailed work. (Gentoo is almost all source code based, has its own build system that’s a bit alien to anything else, and X building is a black art at best, obscure hell most of the time, and “run screaming from the room hair on fire” often…) As I’ve already installed just about every other major release for the Pi (there are dozens of more minor ones…) I decided to give Slackware a try.

Slackware is one of THE oldest releases of Linux. (I’m tempted to say the oldest except Linus must have been running some tools on top of his kernel… yet was that a ‘release’ if it was never released?…). It “has its own way” about it. Slightly surly curmudgeon about gratuitous change (that’s a good thing!) and a tendency to like things that just work, even if you do have to pay attention some times. (i.e. not fond of GUI layers of abstraction that make it easy for a noob to, say, bring up a default network, but make it hard to have a clue what you are doing and / or get in the way of the experienced hand… I’ve had that under other distros with wicd arguing with other network config stuff…)

So it tends to be a bit “musty” in general appearance and action. Yet “it just works”. Yes, some “old friends” of problems are still around where you can trip over your own shoelaces with things like having BIND running and trying to launch dnsmasq (which I did. More below.) Yet they have a philosophy of “release when it is ready, not on a date”. So if something is known broken, it doesn’t release. I like that. It also has a more “old school” build / install process, so I figured “Hey, easy, I’ll just stuff it on a chip to get the mojo working again.”

Then Slackware reminded me that “It’s never that easy. -E.M.Smith” (however easy you think it will be… It’s never that easy…) Yet none of the “issues” was really much. More in the niggling little things range. Like finding that dnsmasq would not start. Initially I thought that a Slackware issue, but doing a web search on the error message, most of the pages were Debian or Ubuntu… so not just Slackware, or even mostly Slackware. Why is the dnsmasq port “in use” so it can’t start? By default BIND is running as DNS and nothing stops you from being stupid and launching dnsmasq too… During the install, I just thought “oh heck, turn on everything” and it gives you that choice… Hey, that is the traditional Unix Way

Jumping to the end, my major surprise is just how fast the combination of Konqueror Web Browser and xfce on Slackware can be. ALL of the “typeahead” lag is gone. Zero. I’ve got the browser open typing an article and CPU usage is about 7%. If I stop typing, it goes to 99% Idle. Unlike a SystemD world where about 10% of CPU goes just to keeping SystemD et. al. idling. Unlike FireFox where it seems to take about 20% of CPU to do nothing, and bounces up to about 100% of one core as soon as I type a word. Frankly, this is just sooo much more comfortable for posting I can not see any way I’m going to “go back” to lags and pauses and “almost good enough”. I have zero complaints about this engine as a posting engine.

On my “todo” list is to see if the xfce / konqueror combo is similarly good on other base systems, like Arch / Gentoo / Raspbian. Basically, find out who has the magic sauce…

END QUOTE.

As of now, I do have a complaint… I could not hit “Save Draft” for some reason… We’ll see if that complaint goes away or not, over time.

In Conclusion

Overall, my impression of Slackware is a good one. A bit “retro” in some of the look and feel department, but solid and fast. Short on browser options, though.

It is an easy bring up system install process, as long as you remember “DO NOT REBOOT!!!” ;-) and has a very fast default desktop / browser combo for general reading and comments to articles. Writing articles on WordPress TBD. (WordPress does many high resource use and PITA things and is not known for code flexibility… I’m still fighting it about not forcing me into the “beep bop boop” editor that I think may be Javascript and a slow pig PITA on low resource machines like the Pi or my tablet). Any “issue” is as likely to be WordPress as konqueror… IMHO.

IFF I can relatively rapidly and painlessly get a Mozilla family browser or Chromium on it, and they stay fast, I’m highly likely to use it as the Daily Driver. (When your home dir is on an external drive, it is VERY easy to move to a new system ;-) You are one mount command away from moved, most of the time. Firefox and Seamonkey / Thunderbird fight over where your mail is stored, but that’s about it.)

Overall, I like it a lot more than Debian / Raspbian. Why? No systemd, faster, and a more general sense that the “Benign Dictator For Life” who sets style decisions has had a reasonable sense about when to “just say no!” to gratuitous changes and fads. Not quite as ‘retro’ and non-changing as BSD, so some of the newer kit is there when needed, yet not running around from fad to fad like much of the rest of the Linux world. In short, I like this guy.

Open issues:

I’ve not yet taken the build system for a test drive. No idea how hard it is to “roll a new one”.

I’ve not yet installed variety browsers or looked at total package richness. There’s a lot in there already (6 GB!), but clearly on browsers it is short, so “some digging required”.

Configuration examination. I noticed, for example, that setting my password had the crypt-text show up in /etc/passwd. Most of the world has this in /etc/shadow now. Since /etc/passwd is world readable, anyone can see the crypt-text. About 1986 when running a Cray site, we figured it would take about 2 TB to precompute all possible password crypt-text values and make password cracking a “lookup”… At that time our 2 TB storage cost $1/2 Million… now it is $59 at Best Buy. That is why the cyrpt-text is now hidden in a non-readable file. Yet here it is readable. Configure option I missed? Who knows… Other users, like root, have their password in /etc/shadow, so the system knows how to do that. It just leaves me wondering how many other “known exposures” have been left laying around for “the experienced sysadmin will know to close them”… discovery.

Compare Gentoo that offers a hardened libraries option…
clearly way over the top aware of security issues.

But maybe it’s just me. Moving into a new system is always a journey in examination of self expectations. There might well be some rational thing I just don’t know about it yet. (Say, for example, they use AES password encryption instead of the Old Way, and I just didn’t notice that…)

Overall, much more usable faster than Gentoo. (no surprise there). Not run into any missing basic tools issues yet (like ARCH where I could not get wget to build…) but a bit light on user space things like browser choices. But those can all be fixed. With time and work.

Subscribe to feed

Posted in Tech Bits | Tagged , , , | 7 Comments

SystemD – it keeps getting worse

There are some things where your first impression is “WTF?”, but on closer inspection turn out to have good features. The point of the pyramid is not so good looking, but the deeper you dig, the better it gets. Then there are other things where the top view is great, but “the digger you deep the deeper the doo”…

My first impression of SystemD was “why on earth do that?”. Digging a little, it became “What the? That’s not right.”. Now I’ve gone further into it via using it on a couple of platforms and had that sinking feeling when “everything you know is wrong, now” and had some inexplicable bad behaviours from those systems. (Things like a directory in which I was residing being apparently deleted and recreated and “kill -9 PID” not causing that Process ID to die.)

So I decided to look a bit more at just what IS SystemD and what all is it doing (now, and if possible in the future given the massive rate of “take over” of functions that it has done so far). I started with The Wiki. Yes, there’s a wiki on it. Oddly, a not too horridly biased one.

https://en.wikipedia.org/wiki/Systemd

Since Wikis have had a tendency to be rewritten if referenced for criticism of The Left, and probably other times as well, I’m going to quote more of it than I would were it a more stable platform. Some bits are out of order, and any bold is mine.

systemd is an init system used by some Linux distributions to bootstrap the user space and manage all processes subsequently, instead of the UNIX System V or Berkeley Software Distribution (BSD) init systems. The name systemd adheres to the Unix convention of naming daemons by appending the letter d. It is published as free and open-source software under the terms of the GNU Lesser General Public License (LGPL) version 2.1 or later. One of systemd’s main goals is to unify basic Linux configurations and service behaviors across all distributions.

As of 2015, many Linux distributions have adopted systemd as their default init system. The increasing adoption of systemd has been controversial, with critics arguing the software has violated the Unix philosophy by becoming increasingly complex, and that distributions have been forced to adopt it due to the dependency of various other software upon it, including, most notably, the GNOME 3 desktop environment.

Right out the gate, the intent is to have ALL distributions “unified” in using it. That is exactly counter to the Free and OPEN Software (FOSS) way. ALL software is availabe for the express purpose of changing over time to become whatever YOU want if YOU (and your friends) put in the time to work on it. Even the kernel. Roll your own if you like. Add some bit of kit you need for your particular hardware or process. Make it a Real Time Kernel if doing things like robotics. Etc. Now along comes this bit of “shimware” that forces itself between “userland” and the kernel and is so full of hooks into everything that it is hard to root it out for your system needs.

A picture showing Systemd as a deep blue layer just below userland isolating it from the kernel and hardware.

Systemd is that deep blue layer just below userland isolating it from the kernel and hardware.

Larger full sized original image here.

So by design this thing is supposed to isolate you and install itself between all other code and the kernel. To take control of all outside the kernel. And to force “uniformity”. That is not The Unix Way, nor the FOSS way, nor even “playing well with others”. Not nice.

Poettering describes systemd development as “never finished, never complete, but tracking progress of technology”. In May 2014, Poettering further defined systemd as aiming to unify “pointless differences between distributions”, by providing the following three general functions:

A system and service manager (manages both the system, as by applying various configurations, and its services)
A software platform (serves as a basis for developing other software)
The glue between applications and the kernel (provides various interfaces that expose functionalities provided by the kernel)

“One ring to rule them all, one ring to bind them…”

Now notice that 2nd line. “Software platform” for “developing other software”? What is this now, a tool chain? An environment for devo? A constantly mutating monster? By design? And that third line “expose functionalities provided by the kernel”? I thought that was the job of the kernel calls and interface? Why do I need a shim stuffed in there in my way?

But it gets worse.

In August 2015, systemd now provides a login shell, callable via machinectl shell.

Graphical frontends

A few graphical frontends are available, including:

systemd-ui

Also known as systemadm, it is a simple GTK+-based graphical front-end for systemd. It provides a simple user interface to manage services and a graphical agent to request passwords from the user. As of 2014 the systemadm program has received little development or maintenance in the last few years, because development focus has shifted to command-line tools like systemctl and systemd-analyze.

systemd-kcm

Provides a graphical systemd frontend for the KDE Plasma 5 desktop. It integrates into the system settings window and allows monitoring and controlling of systemd units and logind sessions, as well as graphical editing of configuration files.

Graphical front ends and a login to this layer between the kernel and ALL users processes? Really? Can you say “REALLY JUICY Attack surfaces!”? I knew you could…

Why on earth have a login (protected how, by whom, with what audit trail independent of that system?) into the guts of a system critical for security logging and authentication? Where does this put my two factor authenticator protection? Sigh. Yeah yeah, they might have done it well, with hooks back out to the external authenticators, but now I don’t know. One of the keys to a breaker proof system is knowing. And, since this is (by design) a constantly moving target, you can never really know.

Now for most folks they just download a binary blob and run with it. Look at the source code on a server somewhere in the “cloud” maybe. This presents and enormous and frankly easy way for “Agencies” to bugger your system. Just watch your wire for a system update check, redirect the request to their server (they do all this part already sometimes) and “update” your systemd with one where they can “login” to it. Now they OWN you. I can’t help but wonder, given the PRISM program, if this is not also “by design”. Make a giant blob nobody but the developers will ever actually read, make key things dependent on it (since they could not corrupt Linus and the kernel, put a layer above it comes immediately to mind) and have it be adopted nearly everywhere. Game and set done, Match underway now IMHO.

Since systemd supports only Linux and cannot be easily ported to other operating systems due to the heavy use of Linux kernel APIs, there is a need to offer compatible APIs on other operating systems such as OpenBSD.

In a September 2014 ZDNet interview, prominent Linux kernel developer Theodore Ts’o expressed his opinion that the dispute over systemd’s centralized design philosophy, more than technical concerns, indicates a dangerous general trend toward uniformizing the Linux ecosystem, alienating and marginalizing parts of the open-source community, and leaving little room for alternative projects. In this he found similarities with the attitude he found in the GNOME project toward non-standard configurations. On social media, Ts’o also later compared the attitudes of two key developers to that of GNOME’s developers.

I gave up on Gnome about a decade back for just that kind of reason. Well, that, and it was a big fat pig of code that didn’t run well and was not, IMHO, designed right. The subsequent flowering of a half dozen alternative “light weight” replacements like xfce and lxde show I wasn’t the only one.

Now one clear “good thing” is that this means by definition systemd must fail at one of the stated main goals. You can’t stamp out all those nasty little variations of preference and “unify” it all if you leave out the world of BSD. It also means that, worst case, I can run BSD and be happy. (BSD is used as the base for the best most hardened systems in the world. China uses it as the base for kylin, I used it as the base for our hardened systems at Apple when I was there, it is widely used in other industrial strength places and by TLAs as well. (Three Letter Agencies…) Hmmm… one wonders if that might be why systemd was designed to not play well with BSD…

On the good news side, there are some islands of sanity unwilling to be lead by the Judas Goat of really fast boot times. LFS Linux From Scratch is not ‘going there’ for the simple reason that things were changing too fast to keep up even if they wanted to, and they didn’t want to much. Slackware uses the BSD style “rc.d” init system from before System V came out from AT&T ‘way back when’ as AT&T tried to mutate Version 7 Unix (BSD base) into something different so they could make money off of a proprietary license. Many of us, then, considered Version 7 the last real Unix… (It was the basis for SunOS and the shift to Solaris as a System V base was not greeted with praise by many of us. I ran SunOS at Apple as long as I could…). Gentoo lets you run systemd, but enables OpenRC by default.

So that’s your “short list” of alternatives. Slackware, Gentoo, LFS, and BSD systems. (And things derived from them).

As of now, I’ve gotten a full Slackware running on the Pi Model 2, Gentoo up to where I need to install X-windows (yet another dogs breakfast of code…) and I’m likely to start a LFS attempt this week. BSD some time later.

Why? Well, I need to “pick one” to concentrate on. I have learned through the years (decades now?) that it is best to do as complete a search as possible “up front” as that removes the most issues later. Oddly, I also follow an entirely conflicting philosophy when “time is of the essence”. I’ve done very many “fast bring up” and “emergency speed” projects. “‘Fast, good, cheap: pick any two’ and you already chose fast -E.M.Smith”… so want it good, you will use excess resources. Once the client understands that, the “Fast Project” method is used. Pick one path as early as possible, start on it, keep someone investigating the other paths, if needed, swap horses mid stream preserving as much of the work done as possible in the transition. For as many parallel steps as possible, ‘rapid prototype’ them for early defect / issue discovery. Propagate that info back upstream. In the end, you do the work 3 times, but with a LOT of time overlap, so finish faster than the alternative methodical linear approach. Rather like modern computer cores that precompute some paths through the code then when they come to the branch / decision throw away the branch not taken… So again, I’m not alone in seeing how this strategy increases speed to goal. But on this project, I am only one person, so there is no excess resource to spend… so the upfront search is the faster line.

I’ve already found that both Gentoo and Slackware are cleaner and IMHO seem faster. LFS is likely to be very small and fast. There is a port of LFS to the MUSL libraries ( a candidate for small fast libraries to replace bloated and slow glibc ) and I may well leverage off of that. There’s a nice comparison of the different library choices here:

http://www.etalabs.net/compare_libcs.html

Now the reason this matters to a systemd discussion is that the “libraries” are what everything else is built upon. IF they are fat, your system is fat. If they are slow, your system is slow. Systemd is locked to glibc and you MUST use it. (Which is likely why the embedded folks at Gentoo just walked away from it. uClibc is essential to making small secure routers on minimal hardware.) Since I want a small fast system that also runs on minimal hardware, I’m likely going to step away from glibc too. (This is not without “issues”. The more recent GNU C compiler has a tie to glibc, so that LFS on MUSL had to cross compile from a glibc based machine…)

http://sabotage.tech/ has a link to
https://github.com/sabotage-linux/sabotage/

This is Sabotage, an experimental distribution based on musl libc and busybox.

Currently Sabotage supports i386, x86_64, MIPS, PowerPC32 and ARM(v4t+). ARM hardfloat (hf) is supported via crosscompilation of stage1, since it requires a recent GCC which we can’t easily bootstrap in stage0 due to library dependencies of GCC introduced with 4.3.

The preferred way to build Sabotage is using a native Linux environment for the desired architecture. It is now also possible to cross-compile large parts of it. As cross-compiling is hairy and support for it is quite new, expect breakage. Native builds are well tested and considered stable.

OK, so it can’t completely self host and self build… Something for later…

But, to the point, systemd force locks you into glibc libraries and that matters. From the chart in the link above, comparision of the library size and of the “minimal C program” complies sizes.

Bloat comparison 	musl	uClibc	dietlibc    glibc
Complete .a set 	426k 	500k 	120k 	    2.0M †
Complete .so set 	527k 	560k 	185k 	    7.9M †
Smallest static C prog 	1.8k 	5k 	0.2k 	    662k 

Yup, THE smallest possible C program ends up 660 kB larger than needed from bloat in the libraries. The entire .so set ends up 7.3 Meg of boat.

Think that matters at run time on small hardware? You betcha!

So one of my design goals is a non glibc based system, for obvious reasons. Yet systemd prevents that.

The only logical conclusion is to dump systemd. Which is exactly what the embedded systems folks at Gentoo did. Good on ya!

Preserving the History quotes

This is from the History section of the wiki, here mostly to preserve it, since it is presently flagged with:

“This article’s Criticism or Controversy section may compromise the article’s neutral point of view of the subject. Please integrate the section’s contents into the article as a whole, or rewrite the material. (November 2015)” the PC Police don’t like the negative waves…

History and controversy

The design of systemd has ignited controversy within the free software community. Critics argue that systemd is overly complex and suffers continued feature creep, and that its architecture violates the design principles of Unix-like operating systems. There is also concern that it forms a system of interlocked dependencies, thereby giving distribution maintainers little choice but to adopt systemd as more user-space software come to depend on its components.

In May 2011, Fedora became the first major Linux distribution to enable systemd by default.

In a 2012 interview, Slackware’s lead Patrick Volkerding expressed reservations about the systemd architecture, stating his belief that its design was contrary to the Unix philosophy of interconnected utilities with narrowly defined functionalities. As of August 2014, Slackware does not support or use systemd, but Volkerding has not ruled out the possibility of switching to it.

In January 2013, Lennart Poettering attempted to address concerns about systemd in a blog post called The Biggest Myths.

Between October 2013 and February 2014, a long debate among the Debian Technical Committee occurred on the Debian mailing list, discussing which init system to use as the default in Debian 8 “jessie”, and culminating in a decision in favor of systemd. The debate was widely publicized and in the wake of the decision the debate continues on the Debian mailing list. In February 2014, after Debian’s decision was made, Mark Shuttleworth announced on his blog that Ubuntu would be following through as well in implementing systemd, despite his earlier comments in October 2013 that described systemd as “hugely invasive and hardly justified”.

In March 2014, Eric S. Raymond opined that systemd’s design goals were prone to mission creep and software bloat. In April 2014, Linus Torvalds expressed reservations about the attitude of Kay Sievers, a key systemd developer, toward users and bug reports in regards to modifications sent to the linux kernel itself by Kay Sievers. In late April 2014, a campaign to boycott systemd was launched, with a website listing various reasons against its adoption.

In an August 2014 article published in InfoWorld, Paul Venezia wrote about the systemd controversy, and attributed the controversy to violation of the Unix philosophy, and to “enormous egos who firmly believe they can do no wrong”. The article also characterizes the architecture of systemd as similar to that of svchost.exe, a critical system component in Microsoft Windows with a broad functional scope.

In November 2014, Debian maintainers and Technical Committee members Joey Hess, Russ Allbery, Ian Jackson and systemd package maintainer Tollef Fog Heen resigned from their positions. All four justified their decision on the public Debian mailing list and in personal blogs with their exposure to extraordinary stress levels related to ongoing disputes on systemd integration within the Debian and open source community that rendered regular maintenance virtually impossible.

In December 2014, a fork of Debian, called Devuan, was announced by a group calling themselves the “Veteran Unix Admins”. Its intention is to provide a Debian variant without systemd installed by default.

In August 2015, systemd now provides a login shell, callable via machinectl shell.

In October 2015, an article titled “Structural and semantic deficiencies in the systemd architecture for real-world service management” was published, which criticized systemd in several areas, including its design as an object system with too many layers of indirection that make it prone to ordering-related failure cases, a difficult to predict execution model, its non-deterministic boot order, implicit state in unit file configurations and its general inadequacy at providing a uniform external abstraction for unit types.

Other than that, no problem! ;sarc/

No, I don’t need my Unix world re-made in the image of Microsoft architecture and sychost.exe design. It is exactly that design approach that drives me away from Micro$oft. Well, that and their participation in PRISM…

In Conclusion

So that’s what I’ve been up to the last day or two. Wandering in the desert of systemd, finding just how much it damages choice, even down to the library system you must run, finding out it is an operating system wanna-be running on tope of the real OS kernel and filtering what the real OS userland can do, complete with its own login… Fighting the urge to wretch in the sink…

From the wiki:

Opinions
“ systemd provides us with a great set of APIs to get done what we want to get done. In ‘earlier times’, we had to do these things for ourselves. Not so long ago, gnome-settings-daemon shipped some system-level services that were a poorly-written mess of #ifdef in order to do basic things like set the time and enable/disable NTP just so that we could provide a UI for this in gnome-control-center. We had to handle the cases for each operating system, for ourselves, often without having access to those systems for testing. These are components that run as root.”

— Ryan Lortie,

So one bit of over reaching bloatware that was trying to do things with the OS that it never ought to have done, needed a more “uniform” layer below it to take the pain and suffering out of their bad decision to be a giant Does-All and not be part of The Unix Way of modular code each doing one thing very well. No thanks. Just deep six Gnome and move on…

“ I understand the natural urge to design something newer than sysvinit, but how about testing it a bit more? I have 5 different computers, and on any given random reboot, 1 out of 5 of these won’t boot. That’s a 20% failure rate. Its been a 20% failure rate for over 6 years now.

Exactly how much system testing is needed to push the failure rate to less than 1-out-of-5? Is it really that hard to test software before you ship it? Especially system software related to booting!? If systemd plans to take over the world, it should at least work, instead of failing.”

— Linas Vepstas,

“ I don’t actually have any particularly strong opinions on systemd itself. I’ve had issues with some of the core developers that I think are much too cavalier about bugs and compatibility, and I think some of the design details are insane (I dislike the binary logs, for example), but those are details, not big issues.”

— Linus Torvalds,

Nice when you find yourself out on a fringe edge, and next to you stands someone like Linus… Yes, Linus, I too saw some of it as a bit insane, and really detest binary logs you can only read with a special program…

With that, I’ve narrowed my base candidates to Gentoo, Slackware, LFS and their derivatives. Most of the rest are either already systemd afflicted, or headed that way. BSD is hanging in the wings as a fallback candidate, but it is so different from Linux that for most folks it would be a bit hostile to install. For me, it would take more work to make it go and make it handholdy enough for others. Puppy has some very interesting build tools, and I’m likely to raid it for some design aspects and tools. (That whole boot to memory and build system kit). As a base, though, Puppy has some issues. It is, by default, single user. It is prone to scatter as they have a lot of neat tools to roll your own, and their own code base shifts over the years. Sometimes Slackware, sometimes others. Flexibility may be nice later, but up front it makes the search space large and the options non-converging. So I’m going to pick bits out of it if they work for me, but not just make another Puppy… (besides, while I like dogs, the constant dog metaphor in everything drives me up a wall…)

As of now, the most likely process I see is a LFS build against glibc (later against MUSL that has been shown to work but needs a cross complie complicated build system), then a 3 way final compare, and a full on system build. For now, my hunch is that the first one will be LFS based as their build system is fully documented and they have an auto generated script that drives it. Essentially, most of the work is already automated if desired and self hosted / contained. Eventually I’d like to get to Gentoo, but the learning curve on their build system will take some time, so it is a parallel task of lower priority. Slackware calls to me with that rc.d BSD flavor and the way it can offer both binary blobs and builds from source, so I’m likely to put fleshing it out for my install ahead of Gentoo exploration. That it only comes with the konqueror browser is a bit of a pain, since attempts to edit an article with it did not end well… I’d rather not start out with the need to port browsers… (Yes, packages are available… but I’d rather not commit to that whole approach just yet).

So that’s where I am now. LFS up next, Slackware being finished out as desired, and Gentoo hanging over my head ;-) I really don’t look forward to slogging through the Gentoo unique installation method AND x-windows in the same bucket…

OK, back to work! ;-)

Subscribe to feed

Posted in Tech Bits | Tagged , , | 13 Comments

Gentoo – First Impressions

I’ve just finished the shutdown of my Raspberry Pi Gentoo installation box.

I did something of a “shortcut” via the directions here:

http://ukginger.net/Gentoo2/

Install Gentoo on Raspberry Pi 2 / 3 + USB Hardrive

This method uses a NOOBS install of Raspbian as the host to install Gentoo onto a USB Hardrive. It is quick & simple, providing a working Gentoo install with the least pain in the shortest time possible.

Please read ALL the instructions before attempting this install

This install method leaves NOOBS and Raspbian on the SD Card as the ultimate Rescue Disc (dual boot).

The install is quick and easy, using the stable Raspberry Pi Foundation Kernel off the SD-Card. No Kernel Compile required.

You’ll also end up with a machine theoretically fractionally faster because it everything will be compiled for ARMv7a_hardFP unlike other distros which are all compiled for the older ARMv6. I also provide an optional Stage-3 compiled for NEON vpf4 to enable all Hardware Acceleration, current-armv7a_neonvfpv4_hardfp.

What follows is How I Did It …. Every instruction has been copy pasted directly from my install. I then re-installed a couple of times following and these instructions to check they are “correct”. If you spot mistakes or can suggest improvements, please contact me.

Over View

The install falls roughly into 3 categories or stages.

Booting Raspbian Desktop to partition & format the USB Hardrive
Download Base Files System, portage Package Stuff
chroot to sort the configs, such as set the keyboard, date, and such like.

Ingredients

Rapsberry Pi 2 / 3
USB Hardrive
8Gb Micro SD card with NOOBS

I did change the mix some. For example, I didn’t need to swap from the strange US keyboard to the UK one ;-)

I also didn’t use their install path of /mnt/genoot as ‘genoot’ is just rude ;-) I just put it in /gentoo and ran with it.

The other thing that I “changed up” was some of the order. They have you make an SD card Raspbian, then use it to format your disk and such. I just used my extant Arch system to launch gparted for the partitioning stuff, then downloaded the Gentoo Tarball into it and unpacked. It really doesn’t matter how the disk partion gets made, or how the bits get onto it…

I used “his” build from here:

http://ukginger.net/Gentoo2/releases/arm/autobuilds/current-armv7a_neonvfpv4_hardfp/

[DIR] Parent Directory –
[TXT] Raspberry-Pi-2.txt 22-Apr-2016 18:53 223
[ ] stage3-armv7a_neonvfpv4_hardfp-20160421.tar.bz2 22-Apr-2016 17:04 218M
[ ] stage3-armv7a_neonvfpv4_hardfp-20160421.tar.bz2.CONTENTS 22-Apr-2016 17:05 4.6M
[ ] stage3-armv7a_neonvfpv4_hardfp-20160421.tar.bz2.DIGESTS 22-Apr-2016 17:05 1.2K
[ ] stage3-raspberry-pi-2.tar.bz2 22-Apr-2016 17:04 218M
[ ] stage3-raspberry-pi-2.tar.bz2.CONTENTS 22-Apr-2016 17:05 4.6M
[ ] stage3-raspberry-pi-2.tar.bz2.DIGESTS 22-Apr-2016 17:05 1.2K

In particular that “neon” tar.bz2 wad. The “neon” processor is a dinky vector unit in the ARM chip that does something like a half dozen SIMD (Single Instruction Multiple Data) parallel math chunks at once. The page goes into reasonable depth about it. The short form is it speeds up sound and video at the expense of not being exactly IEEE spec precision. I’m OK with that on this system. (I’d likely choose the other way if making a Temperature Model Validation system as I’d need to avoid the accusation that I was the source of math drift…)

In general, his directions are “spot on” and you can just cookbook what he says. My only real complaint is that after repeatedly having my fingers type vi commands as soon as I thought to do something, and having “nano” not take well to that, I decided to install vi. On Gentoo that is done via: “emerge vim” (and then waiting several minutes while “stuff happens”…)

Issues

This method of install avoids a kernel build since it reuses the Raspbian Kernel. That’s OK, sort of. Especially just for a ‘first taste’. BUT: It means that I’m running a hybrid system where, as he points out, compiling and installing something like the X-windows system might be a bit off. It also means that the kernel was not complied with all the goodies and flags set for optimal use under Gentoo. I’ll worry about that next week…

So on my “todo list” is “make a Gentoo Kernel using this Gentoo and swap them”. Likely about a day in about a week. Maybe. (Depends on if I actually notice any issues, or not ;-)

At the end of a 1/2 day of adding source packages and building them, I’ve got a command line prompt and a lot of “always there” utilities and stuff that are missing. This is normal. Eventually I’ll make a “build script” for adding all that stuff, too, but just realize the basic install is pretty basic. No X-windows, no browsers, no LibreOffice, etc. etc. That’s why I’m doing this posting from Arch instead.

The build process is not as user hostile as I’d expected, but not something a noobie will love to see the first time. In many ways, it is a like “pacman” or “apt-get” in having a simple command line and easy recipes. OTOH, you get screen after screen of “stuff” that will be meaningless to most folks who don’t regularly compile systems and applications source on Linux / Unix boxes. Yes, you can just ignore it… but then what’s the point of compilation on your box? Might as well just download a binary built for it… (For someone “like me”. the advantage is that I can read that stuff and see what code tosses lots of warnings, that you mostly ignore anyway, and set custom compile flags as I like it. Things like that “neon” and optimizer directives for things like memory vs speed.)

It is longer to add a package. Sometimes significantly. Oddly, most of the added time is in the emerge “setting up to make” steps and not in the actual compile times. The “cc” stages past quickly.

While I’m up ‘relatively quickly’ and that means I can “assess and move on” quickly, I’ve also built in a time deficit of “build and swap kernels” of about 6 to 8 hours for “someday” and potentially can’t get an X-window environment running until I do that. (There is a bit of a ‘here there be dragons’ about compiling X without the kernel swap in the pages, but that might just be ‘reasonable paranoia’ as opposed to real data… so I’m going to try it anyway ;-)

It is afflicted with color. The vi (vim) editor uses “helpful colors” to show you what different lines do, and incidentally make it near impossible for me to read the comments in a file as dark blue on black is, er, a challenge… Your command prompt has colors. The ls has colors… Yeah, I know, I can “fix it”, but just wish it was B/W by default.

The Good Bits

The UK Ginger page specifically shows how to set the compiler flag to use multiple cores. I did, and it did. At times, the usage meter in the upper right corner pegged at 100% for a few minutes. ( I was chroot Gentoo in one window, but inside an Arch X-widow wrapper, so had the meter running ;-)

This is Very Good News. This is the same box that would crash on 100% CPU under Debian. This IMHO shows it is NOT the chip (SOC ARM) that has an issue. It just cranked right on through the big compiles. I intend to do more extended parallel processing tests on Gentoo, as that is a particular interest of mine.

It uses the v7 and Neon bits. Every bit of optimization helps, and starting off by using all your instruction set and all your hardware math coprocessors is a good thing.

It looks like anything you want is available as a source package and is one “emerge foo” away. Just need to make that shopping list and wait while they compile. I now have a ‘someday’ project of finding out how much work it is to duplicate the source archive. In a really paranoid world, you grab a mirror of the source archive using somewhere that is hard to bugger (like a public access point or fresh bought wifi hotspot not tied to your name). Now anyone watching “your pipe” doesn’t know to redirect that ‘fetch’ to their buggered versions. From that point forward, you run builds from your own, private, source mirror and it can’t be intercepted. Not worried about that now, but someday maybe. Yes, you also do all the compares of hashes and validation of keys and all… but it is nice to only do that once, not on each package build.

It seems fast and snappy. Yes, all I had was one terminal open (even after the reboot to live). No testing yet, but just an impression.

I really like that they have you do a “chroot” into it under Raspbian. Indirectly you get instruction in how to be running Raspbian, then swap to Gentoo on the fly… Nice.

I really like they have you install to a USB disk. I used a real USB hard disk, but a stick would work too. Lets you leave the world of “SD card wear and corruption issues” if you like. I also suspect that the whole ‘write to disk’ stuff will be much faster (no more writing a 10kb block to change 1 byte…) There is just something more “right” about having a Linux running from a hard disk with real swap ;-) And likely faster too.

Overall the feel is ‘familiar’. Things like the init system is where it ought to be, and you edit the files I know and love to change settings. “Everything you know is right”, as opposed to what SystemD did to things.

The general impression is that this is a place where some folks with skilz live, and they have kept it tidy.

Going Forward

I likely have a couple of more days before I can launch a browser and post a comment. A long list of things (many unknown to me at the moment) will need to be installed first. This is a “bootstraps up” operation, not a prepackaged Ubuntu Snuggle Bunny. Puppy has me up and working in about a 10 minute install. This is likely to be 2 or 3 days to finally done. BUT, it will be done the way I like it, and with pretty much all hardware optimizations turned on.

For the next few weeks, at least, it will only be an “occasional runner” and most of what I’m doing will be from Arch. Arch is in some ways 1/2 way from Debian to Gentoo. It starts out command line only, like Gentoo, but you are about a 2 minute “pacman -S” away from X and a few more to a browser of your choice. Gentoo is a day or two instead. Raspbian is quick out of the box, but I’ve “had issues” with things like OpenMP being doggy, crashing when I ran all 4 cores to the wall (likely a memory manager issue), and being increasingly SystemD “Experienced Admin Hostile”. I’m just getting tired of “I’ll just add a line to…” turning into a few hours of “WTF, where did they hide THAT config.”

For now, I’d suggest Arch as the “1/2 way to techland” happy medium. Gentoo for the “done systems admin before and I like it” folks. Puppy made a strong showing in the latest build, so somebody was busy fixing it up. The graphics ar less crayon 3rd grade and more colored pencil Junior College Art Class. And Chrome works! It’s the only place on the Pi where I’ve found a working Chrome browser (so far… now that they have it working, the others ought to pick up the fixes quick). It is now my “go-to quick boot to check something” choice. Debian is good for folks wanting to just follow what the most folks are doing. More likely the “advice” you want will be on line somewhere (HOWEVER: there will now be two sets of advice… the pre-SystemD in large measure, and the “SystemD is different” slowly building). Ubuntu? Fedora? Both well made and for folks who like their particular styles. Ubuntu is a cleaned up Debian with Mint desktop added. Fedora is in corporate coat and tie and doesn’t want you messing around changing things from “their way”, yet when I need a tool to ‘just work’ like gparted, that’s the one I booted.

I will tend to use all of them at various times, since this is what I do for a living, and knowing them all is important to me. Also, too, each one has their own flavor, and sometimes you just want one over the other…

Per a Build Base: Right now, the short list is Arch, Gentoo, and LFS. I don’t have a favorite yet. I need to do a LFS build first, so I know what I’m comparing. Next week sometime. Gentoo is clearly in the “Yeah, it works, I can start from here” bucket. Arch is in the “I can probably start from here… where are those sources?”. Arch also may (probably will?) drop support for SysVinit “soon” as they have committed to SystemD (and I’m not); so it is less likely long term. Finally, LFS has sidestepped SystemD saying that it was causing so much change every day it was impossible to keep up (which is sort of my impression). However, I don’t know if ALL source packages are available to build a full service LFS box. Things like LibreOffice and Gimp? Don’t know… So some exploration needed there. For now “Gentoo will do it” and I need to finish a systems build out, then test drive it for a week or two to know keep vs toss.

That’s where I got to. More as more happens ;-)

Subscribe to feed

Posted in Tech Bits | Tagged , , , , , , | 18 Comments