It is now 9:30 PM on Monday. I started down this path sometime Saturday. What ought to have been a “3 hour tour” has taken a bit longer…
I’ll give you the very abbreviated form, skipping the half dozen times I tried slight variations on a theme and had nothing happen differently and the half dozen times I got to do that half dozen over again as I found a bit of hardware had failed…
The Basic Process
I downloaded the various new Devuan 2.0 “ASCII” code name files from a mirror site. They changed where you get them from, so now you get to pick a mirror. One was fairly dog slow, and another blindingly fast. I stayed with the fast one… a .edu site on a weekend.
Then the usual unpacking (unxz) and you get the binary blob that is an SD card image and dd it onto the chip, in the computer and “Bob’s your Uncle” up it comes! Oh Boy!
Problem #1):
It builds a roughly 2 GB file system on your SD card that you then get to manually expand. No problem, just use gparted on another version… EXCEPT, they have gone to the “new” EXT4 that is different from the “old” EXT4 (but has exactly the same name…) and since the journals are different, “old” gparted can’t handle it. Now one could just use the NEW gparted, if they could install it, in the space that’s too small to install LXDE and gparted (g for Graphical..). Basically if you could install it then you could use it to let you install it… Sigh.
I chose not to expand the file system ‘long hand’ at the command line interface… Instead, I used the Devuan 1.0 system, that is my Daily Driver, to tar off the contents, then replace the offending ext4 with an ext3, then put back the tar archive files.
At that time you also get to change /boot/cmdline.txt to say ext3 instead of ext4 also (on the msdos partition).
That let it boot and all was fine. I’d done this with the arm64 64 Bit version for Raspberry Pi.
ARM64 Is Fat and Slow
Well, just like last time, the Arm64 version is fat and slow. Memory gets used up with just 3 open tabs and it’s off to swap land, where swap to SD card is also slow. It does work, but the armhf 32 bit version uses less memory and is somewhat faster to load things (as reading an SD card is the speed limit and 64 bit words take twice as much transfer as 32 bit words for the same number of words). Someday they will have optimized the 64 bit code and done things like us the “thumb” codes to make it smaller and faster. But right now they are still in the “just make it go” stages.
So I decided to use the 32 bit Armhf Raspberry Pi Model 2 code copy instead.
Armhf doesn’t boot in Pi M3
It has always been fine to run the 32 bit op codes / programs in the 64 bit Pi M3. For reasons I’ve only barely figured out, it did not want to run with what was shipped. Skipping about 8 hours of today: I got it to work by just brute force copy of everything from the old Devuan 1.0 /boot into the Devuan 2.0 /boot. Some binary blob in the process isn’t quite right, IMHO.
Now it boots, and I’m happy.
Getting There is 1/2 the Misery!
So what took so long?
First off, for several hours I had intermittent issues with SD cards that would NOT boot and sometimes became unreadable. Even to the point where launching gparted would hang the session. ( I resorted to having a window open as root and in it, I’d launch the command: sleep 5m ; halt )
That way, in 5 minutes, the machine would shut down gracefully if my session hung. Then I’d launch gparted and if it hung, go make coffee. If not, I’d kill the command in the other window.
So what was going on?
Problem #2):
Turns out the little adapter I was using to read / write the micro-SD cards (a USB / SD thing) was failing. At the end of the day it was reliably not writing the Master Boot Record. It took a while to figure that out. I have two of them and usually don’t care which one is used for what. Had to get past that habit. That took a few hours to figure out, just because you don’t really expect a basically passive device to fail. Then you especially don’t expect it to fail in just such a way that the OTHER file systems get written but the MBR fails. I still don’t know how that can be.
Now add in that even once it was replaced, the armhf copies would not boot anyway, and it’s a 2-fer trying to figure out which thing is the issue. The “breakthrough” came when I had a known good chip in it and gparted would not display the information, but moving that chip to the other carrier worked. Eureka!
Then several more hours working out that the armhf was STILL not going to boot the Pi M3 anyway and it was NOT device related…
At that point I went to unpacking the tarball version they also have. Un-tarred it, stuck the /boot stuff on the MS FAT32 partition and the rest on the ext3 partition (all created and “lba” flag set in gparted by hand) and expected “This Time For Sure!”… no joy again…
Problem #3):
That’s when I went through looking at the stuff in /boot (MS DOS partition). It’s a lot of binary blobs, the kernel, and a few txt config files. I chose to just pick up the whole set from Devuan 1.0 and stuff them in. At that point it booted and ran. So “something” in the /boot partition is bogus for the Pi M3.
I need to go back and narrow it down at some point. I’d like to get the newer kernel, so that’s the first thing I’m going to try. Hopefully it’s just the binary blob that’s the GPU boot loader code.
It is quite likely that something in the rest of the system will not work quite right with an older kernel so just running it is a risk, though all the basics are usually OK. Still, the whole idea of the upgrade is to, well, upgrade.
And Now?
Now I’m going to go back to watching The Trump & Un Show on the news. It’s been a very frustrating couple of days, mostly due to things that ought to “just work” not working.
My path “going forward” is for tomorrow to swap in the new kernel, and then maybe check what other bits might be interesting, and call it “good enough”. After that, the usual 2 or 3 hours of adding in all the bits of applications and systems software I typically use (my “Build Script” stuff) and then I’ll start unpacking my home directory archive into this brand new space.
I’d decided that if I was going to be sorting out the various backups of my home directory, I might as well make this an entirely new and full system. Little did I realize that was going to be a “3 day decision”…
But by Friday all ought to be fully recovered, brand new system build, and lots of legacy crap jettisoned. (like all the browser cache). Somewhere along the way this weekend, Firefox, on the current Daily Driver, started to do consistent hard crash on launch. Don’t know if it’s a Firefox thing or a “my cache” thing, but I’m just going to do a new install on Devuan 2.0 and move on anyway.
At present, I’m doing my browsing on the Mac or on the XU4 Odroid. I tried at one point to get Devuan 2.0 to run on it, as well, but it would not boot. Now I’m thinking maybe that was the fault of the USB adapter too. So “someday” I’m going to try again with the known working one and answer that question. I can’t believe that Devuan would ship 2 images that both don’t even boot on the target hardware. (My guess is maybe they tested the Pi M2 image on a Pi M2, but not a Pi M3; and that mode of failure ought not to apply to the XU4… unless they only tested on an XU{something else}.)
But that’s for tomorrow, or maybe the next day. For now, it’s late, I’m burnt out, and I’m going to veg; well away from operating installations and making file systems and dd image moves…
Musings from the Chiefio 
Sorry to hear of your disappointing results of the Devuan 2.0 upgrade. “Some days are like weeks”
I have found that not all downloads are made the same. I have had several failures to get a good build from a download, erased everything and started over with a new download and build and wow it worked! I guess that out of 2gig of bits of download getting everything right is a crapshoot even with error checking. A lot of this computer stuff seems to me to be F.M.when it works, in-spite of what I do rather then because of the things I do.
This “NEW” and improved version of EXT4 that is not really EXT4, what were they thinking? make a deliberate cludge? just because they can.
I have wondered why the RasPi-2 is more valuable to real builders over the RasPi-3, maybe 64bit is a bridge too far in SBC or at least near the limit…pg
@P.G.:
That kind of stuff happens. IF the image is actually not usable in a Pi M3, it will be rapidly discovered and likely fixed in a few days / weeks. MOST of my grief came out of the USB/SD adapter failure. I think it was progressive during the weekend and not always fatal, until it was…
I’ve had that particular device for a decade+, so reasonable for it to finally wear out, I guess.
With it out of the way, the only real “issues” are the EXT4 and “something in /boot”. The “something in /boot” I’ve run into on other Linux machines / OSs before. It seems common…
The EXT4 issue is a bad “own goal” by someone. They made a change to journalling that someone decided was so great it just HAD to be rolled out. To make that easy, old EXT4 file systems will be auto-magically “upgraded” to the new form as soon as it touches a new form OS. All that is just FINE if you have a single system that you upgrade in place and never roll back.
It fails when you have a disk that moves between systems and they are of different release levels spanning the divide. It also (apparently) fails where you have a new EXT4 that needs changing and /or fsck and only the old tools (they do provide a nice automatic nag message that says “You need to install a newer e2fsck” or some such… ) My solution is just avoid EXT4 until such time as it’s ALL the “new format”…
They REALLY ought to have called it EXT5 and let folks choose when to convert, but likely felt the change was relatively minor “just the journal” and not worth a whole new release level hoopla.
BTW, that is NOT a Devuan issue. It is Linux wide. ALL releases using EXT4 are going to have this problem. It is the fault of the folks doing the EXT file system maintenance / enhancement.
In the end, it was not the worst upgrade cycle I’ve ever had. Just not nearly as easy as it ought to have been (and most of that was the $10 hardware failure… and my sloth at realizing the file systems could be written and still have hardware breaking the MBR…)
Yes, sometimes downloads can be broken. That’s what SHA numbers are for. So also on my todo list is just to check that HASH (that ought to be done to assure security and no TLA guy swapped your bits…). So I’ll do that, then try another download. Someday ;-)
Oh, on 64 bit:
At 32 bit you’ve got almost all the advantage possible from longer word length. “What the longer word giveth, byte packing and unpacking taketh away.”. You can get a bit more with 64 bit, but mostly only an advantage on double math. Most of us don’t do double precision math much…
Then mix in Amdahl’s Other Law. The small SBCs are typically shorted on memory to keep cost down. The CPU die is cheap, so you get a few of them, but memory isn’t. Now the bigger word size makes much of that memory less effectively used. Add in the very slow data path on cheap SBCs and it’s a bad combination. They need bigger memory and faster memory paths on SBCs for 64 bit. The Raspberry Pi foundation recognized this when they said the Pi M3 main benefit was the faster clock, not the word size. They were right.
IMHO, the better theoretical design choice is the octo-core 32 bit like the XU4. About the same number of gates and data paths, but more usable on most problems (IF coupled with enough memory… and heat removal).
Oh Well. At $35 / board nobody is really going to care about optimized performance hardware balance in design…
The ARM instruction set has “thumb” instructions that let you use smaller word sizes inside larger word machines. Originally started for 16 bit instructions in 32 bit machines as even 32 bits is starting to have memory issues. Over time, the entire arm64 code base could be optimized to use thumb instructions to good effect and get back speed and smaller code size where the 64 bit instructions are not beneficial. That will take a lot of time by people who care to get it done. Unlikely to happen soon…
It’s part of the loss of care for efficiency. It’s all about everything else now. When folks are mostly interested in dispatch of “dockers” to “virtual machines” they are already a long ways away from efficiency issues…
Sounds like a case of dirty contacts EM. Alcohol (isopropyl) and Q tips, and alcohol (beer) if it does, or doesn’t work ;-)
“Sounds like a case of dirty contacts EM. Alcohol (isopropyl) and Q tips, and alcohol (beer) if it does, or doesn’t work ;-)”
If you have the physical access, the best contact cleaner is…
a pencil eraser.
Easily removes even the “baked on” crap, and rarely damages actual contacts. And common (ie,easy to find)
“the best contact cleaner is…
a pencil eraser.”
Long after I stopped using pencils, I always carried one in my tool kit for just that purpose. Contact leads seem to weather better these days, but when a board starts getting flaky, that is always step one in the debugging process.
Back in the “good old days”when I was a shipboard electrician, we used a ball point pen erasure to clean contacts, you know the white ones, rather then the pink ones on a pencil.The white ones were a bit harder and had a fine abrasive in them, The pink ones were too soft to break the oxide buildup.and would sometimes leave rubber on the contacts…pg
The problem with cleaning the contacts is that the mini-SD card is about as thick as stiff paper so getting something back to where the contacts are located would be a big challenge, as would getting out any ‘crumbles & dust’ created in the cleaning. (Yes, this adapter has a micro-SD slot in it so you don’t have to use the micro-SD to full-SD adapter then put that into the USB adapter … )
Then I’d always be suspicious when using it…
As these things are about $10 and a few months ago when I could not find what drawer / pouch it was in I just bought another one so I could keep on working, I’m most inclined to just pitch it and IF I ever really need a second one, go buy another one new.
Basically I’m opting to be a lazy bum and toss money at it “someday” ;-)
Per Current Status:
Well, after a bunch of mix and match trials, I got it down to three things. First off, shut off the use of graphics acceleration in the config file. Second, use the old kernel. Third, there are two hardware profiles. One is used by the boot step and one is used by the kernel. The one used by the kernel looks to be identical between releases, but the other one changed.
So, I tried it with old kernel and new hardware file and it booted, but the keyboard didn’t work. Swapping to the old hw profile, it worked. HOWEVER…
After installing stuff like LXDE and Firefox, and rebooting, I was presented with a very nice login panel… and the keyboard again was being ignored.
So:
At this point my assumption is that they tested the new boot stuff and kernel on the associated hardware and as the Pi M3 ran on arm64 and the Pi M2 ran on armhf, they did NOT test the cross of armhf on Pi M3 and just said “ship it”.
I’m going to let “ASCII” / 2.0 just sit for a while and see if enough other folks are running it that bugs get reported and fixed without me… and maybe at 2.1 I’ll try again ;-0
I’m happy with 1.0 for now. Maybe in a few days or weeks I’ll try a 1.0 to 2.0 “upgrade in place” instead of the new install and see what happens.
Basically I’m not calling a dead halt on this, but it has entered the land of “needs dickwith factor” and that comes out of limited “play time”. So I’ll play with bits of it on and off, but it gets moved off the production time line.
As Firefox has decided to just repeatedly crash on my Daily Driver (even with re-install and such) and since I need to roll back my home directory anyway, I’m taking the easy way out and just going to do a full restore from about 6 months ago when everything worked. After that point, I’d moved all my data folders off to other media and was just using it as a browsing station, so all I lose is anything I’d bookmarked (little and available in bookmarks file anyway) or downloaded / saved since then (also available on recent backup).
In essence, Instead of a Great Leap Forward I’m just going to do a ‘roll back and recover’.
It has been fun to play with though. But time to move on for a while.
Just to document where I got to, in detail. for 6 months from now when I’m trying to remember what I already did ;-)
All this Magic Sauce is in /boot (on the MSDOS FAT32 partition).
First off, a bunch of stuff in there is just for OTHER Raspberry Pi models or is legal notices. Just to keep it clear to me what I was working on, I moved all that into a new folder named “Ignored”, since on the R.PiM3 it is.
The “kernel.img” file is for the older boards – pre M2 & M3. They use kernel7.img
The files of the form bcm2708* or bcm2709* or bcm2710* are hardware descriptors for the boot loader step of various boards NOT the M3. (Note that the cm3 one is a Compute Module M3 and the 3-b-plus is a model I don’t have yet)
The files of the from bcm2835* or bcm2736* are the hardware profiles for the kernel level to load. Again for hardware I’m not running.
That leaves:
where you can see NEW_2.0 as a folder where I’d stick the new copy to get it out of the way and OLD_1.0 for the similar function with the old version. This let me swap files by just moving them between folders.
I’m running the bcm2710-rpi-3-b.dtb and bcm2837-rpi-3-b..dtb hardware descriptor files, but do note that the bcm2837-rpi-3-b.dtb has not changed between releases.
(I also did a diff of them and they matched).
You can see that I tried the old bootcode.bin, but it made no difference so got set back into this holding space.
All the fixup* and start* files were kept as new and don’t seem to have mattered, just as the overlays were unchanged.
The things swapped out from new and replaced with old ones are:
Which leaves the config files:
The only difference here being the rootfstyp=ext3 as I’d reformatted it to avoid the EXT4 BS.
Here, the very last line was commented out to stop the overlay for faster graphics. It was added in the new one. I left in the larger memory split of 128 for the GPU (old is 64) as it ought not matter.
So that’s the down and dirty on where I got to.
The arm64 boots and runs fine on the Pi M3, but is a bit slow and fat. I’ll likely keep an image of it on a chip and use it a little just to find out if there’s other surprises.
The armhf doesn’t want to install and run right on the Pi M3, but “someday” I’ll give it a spin on one of my Pi M2 boards and see if it is happy there (confirming my testing limited hypothesis) or if it fails there too (saying they didn’t really test it) AND I’ll need to confirm my copy isn’t in some way corrupted in transmission before “going there”.
With that, status documented and time ot proceed to some other tasks…
When I was electronic tech, we used Kraft paper to clean contacts. We could get it from brown paper bags back then. Folded or layered to the right thickness, it might work to clean contacts in SD socket.
When necessary, I used a small screwdriver or knife to clean contacts. All about clean up/contact in the end. Just sayin ;-)
Same goes for those stinkin batteries left in stuff that leaked.
Contact cleaning for deep slots: try WD40 on a piece of card cut to the right size and of the right thickness. For contacts you can see, Gariflex blocks (spelling? Long time since I’ve bought one and they last many years) used to be available. Though actually for toolmakers use in polishing moulds, they are fine abrasive in a rubber block much like the ink erasers, with the difference that different grits are available and the quality is somewhat higher. Maplin’s used to also sell a similar block for cleaning PCBs before etching them. Sadly, Maplin’s are no more…. Also usable in cases of bad soiling are the glass-fibre-filled pens, but they are pretty abrasive and need gentle use in order to retain the Gold as much as possible. For the card adapter, though, the Gold plating would likely be very thin and only really good for a few insertions, so even when cleaned it still may have a problem after a fairly short period of seeming to be fixed.
Well, on a completely different topic from contact cleaning…
Things are going from bad to worse on the desktop front. I’m not sure what all is happening at this point.
So this message is being typed using one of the Cluster Pi boxes on a Devuan 1.0 release that has not changed nor been updated in months. Firefox works.
The Headend Pi that i’d also been using as my Daily Driver, and had updated, has now gone completely “bits up” (like belly up but it doesn’t seem to have a belly ;-)
For a while, FireFox was just crashing very reliably, meaning that it would not get past “restoring your session” then crash. So I moved to the XU4 to do some work from there and post from it.
On the XU4, Firefox also started crashing. Same way / place.
Attempting to take a ‘dump’ from the Sony 8 GB Chip that was the Headend, it worked but then refused to mount again claiming disk errors. An fsck of it kept getting worse and finding ever more bizarre things until it claimed the supperblocks (including backups) were bad. I’m hoping this wasn’t a difference of opinion over the level of EXT4 in use… (The chips are built by default with EXT4 and I’ve not gone back to eradicate it figuring they would be using their own compatible software… ignoring the edge case of looking over a chip on another system and IT maybe ‘upgrading’ the EXT4 to an incompatible level…)
So, in any case, the Headend chip is now toast. Not a big deal as I have backups from a few months back on my file server. (And it doesn’t change much – the home directory being on different media, it is only the OS / software installed)
So, options and wonderments:
Is Firefox broken in some update level and i just happened to slide over it in some update /upgrade habitual thing you do before installing a bit of software? Or is it not compatible with some other bit that changed? Do I need to stay “back level” even from the latest Devuan 1.0 build? For that, I’ll need to establish an all-working base, update / upgrade; then test Firefox.
Are some of these chips just reaching the EOL point where they go POOF! The Sony has been in use almost as long as the Monster USB. MOST of my chips are static, meaning they have something loaded and then just sit in a box awaiting occasional use, but some have been in daily use for a couple of years now… as the Daily Driver or similar. How does one assess the “wear level” of a mini-SD card?
Is the ‘other” Raspberry Pi hardware suspect? It is fairly new ( half a year to a year?) and has not shown problems, but hasn’t been used that much. I just swapped it into the Daily Driver position (and this one to the Pi Stack) a few weeks ago. Not too long before these “issues” started to show up. Yet it had worked just fine from “new” to recently (and may still be fine…). This means I need to take a proven good chip (like this one) and put it in that Pi and see if all is still fine. That also risks blowing this system… (Maybe I’ll just use this one to ‘spin’ a new chip image onto a known good Chip… ;-)
And folks wonder why I have more than a couple of posting options / boxes… Never out of action, just sometimes the 3rd level down in choices ;-) Still have the Mac and 4 more SBCs in the Compute Cluster IFF I need to “go there”…
Ok, the “where I’m at”:
This is now a proven working rig. I build back out from here.
The headend Sony Chip is “presumed dodgy until proven otherwise”.
The “new” headened Pi is going to be loaded with a known good image and re-validated.
The data archives stay isolated from those testing behaviours (just in case there’s some kind of “data VD” going on with crap migrating in the systems…)
The XU4 will get an older chip image put in and FireFox tested. Then an update / upgrade and tested again. IF it goes from good to bad, that’s an answer. If not, then what’s killing FireFox?
Sometimes I really wonder just why computers will work perfectly for months then suddenly all sorts of things “go off the rails”. Often unrelated things. Sometimes only with a relation that’s discovered later. (FireFox updated to borked, or power has too much ripple, or even too many Cosmic Ray Bursts…) But, for whatever reason, sometimes It’s 3 A.M. and you find out you are not yet going to bed… But remember: “Tomorrow is another day!” (swell of uplifting music and sunrise over Tara… )
Well, this re-validates the 2nd Pi M3 B.
It’s working just fine with a known good mini-SD card / OS Image.
Next up I think will need to be a check on vanilla Devuan 1.0 with an update / upgrade to present, then testing FireFox.
At that point, presuming it works as it ought, we have known good hardware and proven to work OS / Firefox.
That will just leave “Failing chips / file systems” or “something corrupted a Firefox Config file in my home directory”. (Or the ever lurking “something God only knows what…”)
I don’t see this as data / config corruption issues as Firfox also crashed when I tried to run it in a “root” login (so different home directory and pristine use).
So most likely it’s just that Firefox “has issues” in a non-systemd port that were not caught prior to some upgrade (as it spans two very different OS copies – a straight Devuan and an Arbian-to-Devuan uplift). It would be unlikely, though possible, that both systems had slow chip file system corruption that caused similar failures of FireFox on the same quarterly time interval.. But I kind of doubt it…
So, next up, re-spin a new Devuan 1.0 install, test Firefox, update / upgrade, test Firefox.
Well, if anyone is following this story…
I’m now posting from a restored “Headend” image with my old home directory. Things are just a few months back dated ;-) but everything works.
I still need to test the Sony SD card and test an old vs newly updated Firefox, but at this point what I needed to get working, is working. From here on out it’s just curiosity and working to prevent a repeat surprise.
At this point it is quite clear that it was not the Pi hardware, as I’m running on that one now. It’s also clear that there are two remaining issues that may well have an intersection.
1) The Sony mini-SD card tossed it’s cookies. It did take a reformat, so maybe it was just long slow file system corruption. EXT4 does not do an fsck at boot, it just checks the journal, so if some bits flip, it will not holler about the degradation. A slowly corrupting file system can cause all sorts of grief.
2) FireFox is sensitive to what’s in various files and paths. So it could be it was just borking over the corrupt file system. Or we do have the potential for a release bug, so I ought to rule that out.
I’m going to keep running this particular system / build until I’m comfortable fully testing a roll forward is not going to recreate the problem(s).
At this point my strongest suspicion is that running a jounalling file system (double the writes) on an SD card with a live OS including swap and /tmp probably has a high enough “wear rate” that it needs some addressing… At least some kind of periodic actual fsck and certainly more frequent whole chip copies.
Sidebar:
My habit of just rolling forward a copy of my home directory to different media from time to time has also paid off as most of everything is in this older copy of it. (This chip mounts a copy that was on a hard disk partition, not the USB dongle). So now mostly it is just compare the last USB backup and make sure I’ve got all that can be got that isn’t here already… I didn’t have anything critical there (mostly the odd downloaded paper or screen shot or some images of software downloaded) with a fair number of saved images that I used in postings (in case the link to the source went stale some day). So basically I’m not worried. Inspection of the Desktop I don’t notice anything in particular that I’ll miss ;-) It sure looks the same.
Well, I think it’s time for a “Beverage of my choice” as it’s been a long night… Coffee with “the works” and a Laughing Cow creamy cheese wedge sounds nice ;-)
Firefox seems to have issues for me. Every “upgrade” makes things worse. It seems to plug up memory by seizing blocks and not releasing them as the chore is completed. I must close it out from time to time on this Intel-Microsoft box to get it back into operation. Your SD card system must really get hammered by Firefox that thinks it is working a hard drive. It is good to hear that there seems to be no corruption in the basic system. Sounds like you have earned a day away from the keyboard…………….at least an evening. Pulling an all niter is above and beyond duty…pg
I have in recent months done the same, switched settings so it clears everything on closing and then periodically close it. Although suspect the real culprit is when I leave a tab open to some web page that has lots of click bait advertisements and does a lot of analysis of user actions.
On those type sites I open them as a new window and close it instead of leaving open as a tab.
And the answer is looking more and more like the Sony micro-SD card is failing.
I reformatted it, and that worked. Then dd copied a system image onto it, and that worked.
Then I decided to go for a completely clean install of Devuan 1.0, so dd copied the install image onto it. Partition map blowen and it looked like a big bag of unallocated bits.
Formatted it again and it took a file system. Did the dame dd copy command and it gave me a bootable system. Booted fine. started to apt-get install stuff. LXDE, Firefox…
It now looks like it is hung near the end of that series of updates. Claimes to be unpackig some user guide, but it’s been a loooooonnng time. We’ll see when I check on it again in 1/2 an hour.
So it may just be that chip, which was the Headend Daily Driver has lost it’s marbles.
OK, I live with that… But there really needs to be a better way of identifying what chip is going marginal / failing.
@P.G:
Well, it wasn’t really an all nighter… I got to bed about 4 AM and did get about 4 hours, then an afternoon nap ;-) I’m a bit of a knight owl anyway …
Yeah, and FireFox can have 350 MB to a GB of cache files it constantly fiddles (all those ads that keep changing ;-)
That’s part of why I put the home directory on a different file system / device.
During this recovery I was reminded that I had at one time moved high use file systems onto small real disk partitions. As a sidebar on this recovery I’m plotting to do that again and see if I can get to just having minimal SD card stuff. It also looks like they have expanded the boot choices in 2.0 so that you can boot form USB, disk, PXE and maybe more. So it may be that a direct hard disk install is possible. I’m going to check on that.
I had done one install where /boot stated on the msdos partition but cmdline.txt was changed to point to everything else on a disk. It worked nicely.
The thing that makes this take so long is that I just stuffed whole SD card binary images onto a backup disk. that means that to restore a 32 GB mini-SD card with maybe 5 GB used, I have to copy 32 GB from nfs mounted disk to SD card… About 1/2 hour at a shot. Then do that a half dozen times to try things ore change releases and it’s several hours later…
At present, I have my home directory, swap, and /tmp on a hard disk. That gets a lot of wear off the SD card and speeds things up. I intend to move /var next…
I really like Opera on the Tablet (which is back in service now, since I got a new charger cable…) and Chromium isn’t bad either. I’m ready to swap to some other browser on the Pi… as soon as something complete and reliable shows up in the repository… A lot of browser guys just don’t want to port to ARM chips thinking them too slow. But yeah, new browser is on the desired list.
@Larry:
When I’m pointed at my ad blocking DNS, the memory usage rate drops a fair amount. I think constantly changing ads possibly with some kind of ‘keep me resident’ flags set might be a big part of the problem… Especially the ones that run a video… It is also a proxy server, and when using both the DNS feature and the caching proxy server, things speed up a lot… I can also set the cache lower on the local computer as the proxy is caching things already.
Well, enough for now. I’m getting sleepy ;-) So I think I’ll call it a night and pick up further pristine system build work tomorrow…
Well the slow unpack finally finished unpacking. ( I went in to shut the system down and it was done with the instal…)
Launched Firfox on a brand new Devuan 1.0 that had the update / upgrade done. Firefox proceeded to crash tabs, then the whole browser.
Looks like some “last update” to Firefox has broken it. So no updating to newest or your browser will break. I’d made a local apt files server some months ago (as the target in /etc/apt/sources_list) so I’ll just point my boxes at that for a while. It was from before whatever was just broken in the updates… So I can get ‘almost new” without hitting this bad level.
There’s some way in apt-get to tell it to backlevel a single program, but I don’t know how to do that yet… haven’t needed to.
So, OK, one more mystery solved. The problem with browser crashes is orthogonal to everything else and it is in the recent release of it.
Maybe we are looking at this backwards. The “beercan computer” should be a dumb machine that barely functions on it’s own without the “personality chip” that makes it special. the SD chip is closer to the SBC then the USB and so it is faster memory IO. Suppose the SD does all all the temp files stuff and the USB drive carries the all OS stuff that makes the SBC do your requirements. That would make the SD chip part of the disposable computer and the Thumb Drive “your” personality to be added to it. Just ruminations with the morning coffee…pg
@P.G.:
You can already get SBCs with the storage built in. Those are eMMC equipped boards.
I paid the $20 or so for the one that attaches to the bottom of the XU4 Odroid.
It’s a little faster than the SD, but harder to reconfigure (different commands to load it, physically you have to take the board out of a case to remove it). So mostly I’ve not used it. It came with Ubuntu pre-installed and worked well, but my first attempt to change the OS on it left it non-bootable and I’ve been slow to go back and learn a bunch of “how to do it right” while I’m still changing the OS often.
So what I’ve done is similar to what you suggest. My “home directory” tends to live on something where I can “unplug and walk away”. Until now, that Monster USB drive. Now they DO wear out, so partly I was just testing how far I could push it. The answer seems to be “a few years” (depending on how often you re-write things – for me somewhere over 3 years. Now I know and I’m willing to pay $10 to $20 a year to change it every 2 or 3 years… before it fails; then use the ‘old” one for things like moving stuff between systems where when it fails I just toss it and use another one as the “traveler”…)
There are folks who boot the entire operating system from a USB drive. I’ve done it. (Even have a Centos 6.x stick and a Knoppix stick for Intel PCs hanging on a hook next to my desk just in case I get called to work on some PC). The ideal way to do it is a “live system” build where it creates a RAM disk in memory and loads the most active bits of the OS into that so USB flash wear is minimized. That’s the “magic sauce” that makes releases like Knoppix “special”. I’ve looked at how to do it, and “someday” it will be my end state. (For now, though, I wanted the high wear rate to get a handle on longevity and risks…)
So, putting it all together:
In an ideal build, the operating system of choice would be a “live build” that loaded from eMMC. Then it would only “save state” to an external USB device (rather like the way Puppy Linux lets you save a state image, or some Knoppix builds let you do that). Now the “Beer Can Computer” is entirely stand alone, boots a generic embedded OS that only gets an update if you make one happen; and at boot time creates a RAM disk that is what actually runs. Only loading / saving individual account / state information from the USB dongle (that can be quite small these days, about the size of a fingernail…) Now the USB gets VERY low wear rates (personal data highly safe) and the eMMC mostly just gets read at boot time (also low wear rate) and the high volatility stuff all happens in a RAM disk (so the build is tuned to be memory efficient as 1/2 the memory will be taken for the RAM disk – also a squashfs file system is usually loaded so that memory space is very efficiently used)
Oh, and worth noting that that systems with built in eMMC and an SD slot could have the OS load from eMMC and the personal files on the SD card. Then to remove your private data from the system you just yank the SD card…
Take a look at any of the “boot from CD or USB Live Systems” to see how it works. The granddaddy of them all was Knoppix I think. Puppy does a good job of it too. Most systems now come with such a build as an option; but I’m not sure if they do it as well. (Ought to though, as they mostly just modeled off what Knoppix et. al. already had done).
It’s one of those Round Tuit things for me. To eventually make a Devuan Pi Live System build and run it in RAMdisk. That was why I played with /usr from a squashfs file system some year? ago. First step of getting ready ;-) Yesterday I laid out my file system plan for this next build, and it had squashfs for a couple of file systems. Haven’t got to the “load it into RAMdisk” step yet, though.
Oh, and there is an “overlay file system” trick to let you put temp writable stuff overlaid on the static actual filesystem. So most of things like /var are static on USB but the overlay is in RAMdisk and lets it look writable… Most used in CD based live boots.
I’ve unfortunately gotten enamored of putting high activity file systems on a real USB disk, so mostly played with that the last year or two instead. It’s a great way to speed up the system and reduce SD / eMMC / flash USB wear. At present, for most boots, I have an old USB Disk plugged in and /tmp gets mounted from it along with swap. That moves all the temporary stuff off the SD card. I’ve also, at times, used partitions for /var and /usr as well. For one build, even / was on real disk and the SD card only had the MSDOS partition stuff with cmdline.txt pointing to it (change that /dev/mmcblk0p2 entry to /dev/sdXY where X is the disk letter (usually a) and Y is the partition number) . Then your SD card gets essentially zero wear and you can just unplug your disk and go with everything on it. But harder to hide in a coin purse ;-)
So yeah, you have the right idea about splitting things up, just that there’s a dozen ways you can do it and lots of fun choices. Your particular goals making the choice. Speed? Put stuff on real disk. Privacy? Write your stuff to an encrypted USB stick that only gets decrypted at boot time, leave a generic OS as “cover” on the SD card. Secrecy? Make it a TOR equipped build. Low wear? Live boot system. etc. etc. Mix and match as desired…
This is a live system built on Devuan:
https://sourceforge.net/projects/exegnulinux/
So that’s the kind of thing that needs doing to make an “embedded” live version for the Pi.
This one claims to be a “DIY” oriented live system. The idea being to give you the tool kit, I think:
https://sourceforge.net/projects/linnix/files/
But again, PC oriented. But essentially take their kit and apply it to the ARM chip build.
These folks making a “Tails” version free of systemD:
https://heads.dyne.org/
Descriptions of them and more at:
https://devuan.org/os/partners/devuan-distros
So mostly it’s not “inventing something new”, just taking what’s been done and repackaging it. Combinations, not invention.
This “upcoming” distribution is a good possible too:
“Just” needs a retrofit to the armhf code base…
After yet more testing:
The Devuan 2.0 armhf raspi2 image is broken. It has something wrong with the MBR / partition map. I can dd other images onto the Sony chip, and they work. No matter what chip I put the 2.0 PiM2 image on, it does not boot.
I’m going to try a new download and SHA check & copy compares.
I may revisit the tar version, but my last attempt with it (hour? ago) also didn’t boot. If I get ambitious, I’ll try it in an actual PiM2 to see if it is an M2 vs M3 thing…
Well, the new download directly from the Devuan site matches the one I’ve got. So it’s a good copy. (the armhf Raspi2 one).
I’ve installed the arm64 release on the Sony card (using it to post this). So now I’m doubting it was the chip after all. Repeated dd of the armhf raspi2 image to various chips all end up with the MBR / partition map missing / wrong. You can mount the file systems sometimes, but it just won’t try to boot. Gparted shows a bit of data at the front of the chip that is empty / unassigned.
Running a browser in the 64 bit version is sometimes painfully slow. It’s OK at the command line as a general purpose work station, but the browser performance is not something I’d want on a day to day basis.
So I’m going to continue using the back level armhf version 1.0 when I need a browser. This one I’ll use for general software and files work. (Moving data around, sorting out what to keep, formatting partitions…)
Using xfs requires that you do an; apt-get install xfsprogs
Then you can not boot into an xfs partition as the boot loader doesn’t understand it. So I’m using an ext3 partition for the root “/” partition. I’m going to play around with what system files can go on an xfs file system (like maybe /var?)
Using xfs as my home directory is going nicely.
So that’s where I’m at tonight… I can easily get the 64 bit version to run, but find it slow and fat. Even on the chips that were unhappy with the armhf 32 bit image. All this leads me to think it’s just not a properly made armhf Raspi2 image or is not suited to run on an Pi.M3 (thought I don’t know how you could make that happen… )
Hopefully Devuan will catch clue and re-spin it.
Pingback: My Magnificent Bastard System | Musings from the Chiefio