I’ll not be decorating this posting with links simply because I ran through a lot of them and it would take a couple of hours to track back and select / sort / enter them. It’s almost entirely the conclusions that matter (and if you don’t agree with them it’s easy enough for you to Dig Here! and find any evidence you like).
First off, the Good:
I bought a Seagate 8 TB disk at Costco for $129.xx and installed it on my Raspberry Pi. It is much smaller than the 2 x 4 TB MyBook disks on my file server and it is USB 3.0 speed. That is nominally $130 / 8 = $16.25 / TB.
My God Man, at that price what’s not to like? It’s hardly worth throwing out the data trash when you can just move it to a “someday” folder and sort it later or not… just call it a “deep backup” ;-)
Then, given some of the recent turmoil in the land of Linux: what with Linus on “sabbatical” having been badgered into self doubt by the SJW Mob, the SystemD crap from RedHat infesting too many releases, then IBM buying Red Hat and Microsoft suing companies using Linux for “patent infringement” (but not willing to share just what patents…) all while buying a seat on the Linux Foundation… well, I decided to take another look at BSD.
At present, all three of the major BSD forks are running on the Raspberry Pi M3. Two of them on all R. Pi sizes 1, 2 and 3. OpenBSD having now shown up on the Pi M3, while NetBSD and FreeBSD have been there for a while. There has also been some progress on making Xorg and LXDE / KDE / etc/ work more easily and be less painful to set up and configure. Part of that being HDMI to TV Monitor like devices being more of a standard (fewer knobs to twiddle to get something to work with every single wacko display you might have). There is now a clear realization that the ARM processor and Single Board Computers are here to stay, there are millions of them, and it is worth supporting.
That 8 TB disk is really 7.2 TB to gparted and 7.5 to the df command (no, I don’t know why they do not agree). That makes it closer to $18 / TB or with California Sales Taxes almost $20 / TB usable. OK, still not going to break the bank, but not the same.
Then, that USB 3.0 would be great if I could use it. Search on “USB 3.0 fail” or similar like “USB 3.0 Fix” and you find all sorts of articles about USB 3.0 failures and flaky behaviour on things from Windows 10 to Odroid XU4 to, well, lots of stuff. Overall, USB 3.0 still “has issues”. Near as I can tell, the one SBC I have with 3.0 ports on it (the XU4) still does not have it working reliably. Hardware? Software? Fundamental failure of design? It would seem nobody knows yet. So it is plugged into the USB 2.0 port of my Raspberry Pi M3 (and I’m thinking it isn’t a 3.0 NOT because the R. Pi folks are dolts and slow but because they refuse to ship it when it doesn’t work right yet…)
In the various ARM BSD releases you find all sorts of caveats. First off, the on-board WiFi doesn’t work. Dongles do. OK… no driver for the odd chip the R. Pi used and nobody writing one at the moment. Not a big deal, but limiting. OpenBSD only runs on the Pi M3 (why bother with a v6 or v7 instruction set port… just go with the less proven, debugged or stable v8 instruction set -CURRENT port…) so NetBSD or FreeBSD look more stable and with better debugging at present. Basically it is still a bit of a Hackers Paradise and not something easy for Joe Average to just set up and run. Yes, you can “cook book” it, but as soon as something goes bump in the night, well… So likely still another year or two for mainline uses.
I’ve been copying my 8 TB (nominal – 7.64 TB formatted) LVM volume group over to the 8 TB Seagate drive. First off, just to have a clean full backup before I go in and start tossing out trash. Second, I don’t really want to be using LVM.
Yes, I know all the reasons “The Experienced SysAdmin will want LVM” since a Logical Volume Manager lets you glue on new capacity or swap out disk with relative “ease”. HOWEVER: Having had to recover the LVM group to a different SBC when the file serve uSD card bit-rotted, let’s just say suddenly installing and configuring LVM was not fun. Furthermore, at the 4 TB / volume size, doing all those loverly maintenance whiz-bangs over USB 2.0 takes forever so I’m just not going to do them anyway. Then there is the PITA of how to back up a 6.5 TB (used) file system. GAK! The experience was just not what I had expected.
So easier to just get a new 8 TB single disk, copy on to it, and have a second copy on the old LVM group until such time as it isn’t needed (then maybe get a second one of these monster disks… FWIW, it’s about 8 x 4 x 1 inches in physical size… about like a small book.) So that’s what I did… and am doing… and will be doing for a couple of more days…
I’ve had the copy process running for a couple of days. I’m not yet 1/2 finished. This is likely to run 5 to 6 days for the copy. Call it 1 TB to 1.5 TB per day. I’d say it is slow but really a TB is very very big. All at USB 2.0 speeds because USB 3.0 isn’t ready for prime time on SBCs. Sigh.
The “good news” is the R. Pi (on Devuan 2.0 straight from the maker) has been rock solid and reliable. It had run 49 days straight when I shut it down to do some testing on other boards prior to bringing it up with the LVM cluster on it. It has copied over 2.8 TB of data (so far) without an issue. I’m typing this on it as it does the copy. (It is ‘D’ for disk wait in Htop so CPU to spare – USB not so much…)
Then some of the BSD ports use the v6 instruction set. You know, the old Pi Model A & B one. Often with only “soft float” math. So not a nice 64 bit instruction set and ignoring the math co-processor. Not going to cut it in terms of performance or efficiency and a real PITA for any science / math stuff or encryption. Yes, I know, smaller instruction width so less memory used and the Pi is ‘thin’ on memory while the other ARM boards sometimes have less and may be missing a math co-processor so “one size fits all”… IIRC it was NetBSD that was this way (as they try to run on all hardware in the world even the very small ARM boards). FreeBSD was mostly v7 instruction set (Pi M2 the older and can run on Pi M3 and the new 64 bit Pi M2). All that argues for using FreeBSD for now and doing your own “build from scratch” if you want lots of speed and things set to use all the Pi hardware. Not exactly Joe Average friendly.
That said, I’m likely going to do it. Why? So at least one of my devices is running BSD, I’ve got the experience base up to date, and whatever happens with Linux I can easily slide off to something I really love and without a pause. Besides, the Devuan Releases “just work” out of the box so I need somewhere to put my Tinker Time ;-)
One of my R. Pi M3 boards arrived with a failed WiFi chip / system on it anyway (why spend $10 of shipping and $50 of time to get a $35 board exchanged… just suck it up and use if for non-WiFi things). So by putting BSD on it I’m not losing anything. It will likely become a generic “server” for experimental infrastructure stuff. A software build box with some TB of disk too. A place to archive my tech collection ;-) Even if it is a bit slow on math at the moment that won’t matter too much.
Once the LVM copy is done, then my main file server store for temperature archives becomes A Single Disk and can move anywhere. I’ll assess the LVM group and perhaps do an “update scrape” using it. I last did a data scrape over a year ago (maybe 2?) when CDIAC was having a “going out of business” sale notice. I need to check in and see what archives died, and what changed so much that an “update” would wipe out my deep archive of the older state.
Basically there is a long slow “unscrambling eggs” process prior to a data scrape update. Eventually I’d like to recover the 2 x 4 TB Western Digital MyBook disks from the LVM group, but “we’ll see”… Things where just MOVING the data take nearly a week are not things that go fast manually assessing them.
It’s not yet time to make a jump to BSD and Devuan is still running stable, true, and clean. I’m going to update my kit to Devuan 2.0 as released (it has been a stable desktop for some time now) and then just forget about all the other SystemD infested releases. Move their archives off to “junk disk” and set it in the corner ;-) I’m likely to keep a working copy of Puppy Linux “just because” – it is small, works well, and has a different build approach. I want to get better and understanding their build process. I don’t know if they have gone over to SystemD or not, though.
But other than that, I’m settled on Devuan and with an experimental BSD “as time permits” The bits of hardware I have that do not have a Devuan port ( 2 x Orange Pi One… all the OTHER Orange Pi have ports, sigh) will stay Armbian with a Devuan “uplift” until a pure Devuan is available. FWIW, I got the new Devuan 2.0 to boot on the Odroid XU4, so “moving my stuff” from the older chip with Armbian on it “in the works”.
It will be a nice reduction in complexity to have basically one OS running across almost everything. Note that my “Interior Router, proxy server & DNS” is going to stay on Alpine. It is a router oriented distribution that just runs great on small hardware and the Pi B+ is very small hardware having only one core at 700 Mhz and v6 instruction set. It is rock solid anyway. It has “just run” for years now and only gets shut down when the power fails.
And, I can get started on that stuff in just a 1/2 week when the data copy to the 8 TB disk is done… grumble…
If anyone knows of an affordable SBC with stable and reliable USB 3.0, well, let me know. I think I’ve reached the point where it matters to me… and Christmas is coming so I can start dropping hints ;-)