My Magnificent Bastard System

After a few days of banging my head on the wall of an amrhf (32 bit) Devuan 2.0 release that just Did Not Run on my Raspberry Pi M3, and finding the FireFox on the arm64 (64 bit) Devuan 2.0 release just painfully slow, over the last 24 hours “I’ve created a monster!” ;-)

Essentially, I took a page from the first days of 64 Bit arm systems when the Linux kernel was new and userland was basically not ported to 64 bit.

I took the 64 bit install, and shoved it onto a mini-SD card. Then I made another partition where I could copy all of the / (root) file system of the armhf 32 bit version. Essentially, you keep all the stuff on the MSDOS partition from the 64 bit install (kernel, cmdline.txt config… etc.) and then change the cmdline.txt so it points to the partition where you un-tarred the 32 bit userland.

The one “tricky bit” is that kernel modules (loaded for things like network device drivers) are located in /lib/modules and you can’t run 32 bit modules in a 64 bit kernel. So you toss the 32 bit and copy in the 64 bit. Then you can do your first boot.

That’s basically it. Runs a champ!

Now I also decided to complexify things mightily while I was at it. At first I did the install into an 8 GB chip, then decided it wasn’t big enough for 2 whole systems, so moved to a 16 GB chip, but only AFTER having installed both, so I couldn’t effectively make the desired file system larger… which resulted in my making 3 more file systems on the chip and moving /var /usr and my home directory onto them.

Yeah, a bit messy, but lets me see just how big each one is with a df command AND sets me up for moving them onto a hard disk mounted partition OR using a squashfs file based file system for /usr to make it resistant to changes. (Once I’m done changing it, that is!) Just for kicks, I also made the added /var /usr and /home/chiefio partitions as xfs file systems. (So you must apt-get install xfsprogs on the system where you grab a copy of userland so it will be there at boot time…)

Here’s the final df output:

chiefio@devuan:~$   df
Filesystem     1K-blocks    Used Available Use% Mounted on
/dev/root        1743136  211904   1441020  13% /
devtmpfs          438328       0    438328   0% /dev
tmpfs              88764    1596     87168   2% /run
tmpfs               5120       4      5116   1% /run/lock
tmpfs             177520       0    177520   0% /run/shm
/dev/mmcblk0p1    130798   35754     95044  28% /boot
/dev/mmcblk0p3   4062912 2854276    998924  75% /arm64
/dev/mmcblk0p5   3135488  596576   2538912  20% /var
/dev/mmcblk0p6   4184064 1692296   2491768  41% /usr
/dev/mmcblk0p7   2073600  574264   1499336  28% /home/chiefio
tmpfs             443800       0    443800   0% /sys/fs/cgroup
tmpfs              88760       8     88752   1% /run/user/0
tmpfs              88760      16     88744   1% /run/user/1000

The /dev/root file system is the /dev/mmcblk0p2 slice and that is where the basic armhf file system is installed. I put the arm64 root in /arm64. It isn’t used, I just left it there for reference. Also, by changing the cmdline.txt file I still could boot in full 64 bit mode.

Here’s the cmdline.txt:

dwc_otg.fiq_fix_enable=2 console=ttyAMA0,115200 kgdboc=ttyAMA0,115200 console=tty1 root=/dev/mmcblk0p2 rootfstype=ext3 rootwait rootflags=noload net.ifnames=0

Note that I’ve made the file systems ext3 (instead of the default rootfstype=ext4) and you just change that root=/dev/mmcblk0p[2-3] to either 2 or 3 to change which userland boots.

When you install more software with apt-get, it caches the new programs .deb package in /var and then the actual install happens into /usr (usually). So both of those are where the growth happens as you fill out a system with all the “stuff” you want in it. When done configuring: /usr is relatively static (as all the ‘variable’ stuff was moved to /var some decades back). That means /usr can be mounted read only or even as a squashf file system on static finished configurations. (A minor added security measure).

Then /boot, the mmcblk0p1 partition on the chip, is the MSDOS FAT32 partition with all the boot loader code, kernel, etc.

Assessment

Well, it’s only been running about an hour now, so this is very preliminary.

First off, FireFox works (yay!) unlike in the last update to Devuan 1.0 or the too painful to use arm64 Devuan 2.0 install. Not super fast, but fast enough. I’m composing / posting this article using it.

It is a little slower than the full armhf system. Not much, just enough to notice. Likely as the 64 bit kernel is not as highly polished as the 32 bit and is moving 2 x as much data for every word fetch. Still, it’s better than nothing (the armhf failed install) or the fully fat and slow all 64 bit.

I’ve not seen any technical issues from running (yet…) and everything seems to run stable and well.

I’m going to be using this as my basic test drive workstation for Devuan 2.0 for a while. I’ll sporadically swap to the Devuan 1.0 Daily Driver for “compare and contrast”. Just to see if my speed perceptions are accurate, or being shifted by subconsciously comparing to the XU4 or Mac ;-) Basically, I’m going to keep myself calibrated as to what the experience is supposed to be.

This particular chip did NOT impress me as being the fastest when I was copying GB onto it. Also, having it chopped up into 7 file systems and putting ALL I/O load onto it is also likely less than ideal. When I last did a “file systems on real disk” test, some year+ ago, that made a BIG difference. So for the rest of tonight and on into next week, I’ve got a 1 TB USB disk I’m going to be formatting and setting up to take a load of the I/Os and see what happens to speed then.

In Conclusion

I’m quietly pleased with myself for making this hybrid 64 bit / 32 bit system. It isn’t all that hard, but it does take a certain attitude, and understanding, to think of “going there”. But that whole necessity thing was kind of in my grill. IF I wanted to test armhf Devuan 2.0, I either had to wait for someone to fix the released copy (I’m pretty sure now it wasn’t me causing it to not work), try it on a PC instead of an SBC, figure out what was wrong and fix it myself, or ‘get creative’. I chose creative as it’s more fun ;’)

This also lets me add all sorts of software without much worry as to what running on a back-level 1.0 release kernel would do (remember that I first got armhf Devuan 2.0 userland running on an older kernel here: https://chiefio.wordpress.com/2018/06/12/devuan-2-0-ascii-release-r-pi-install-tale-of-woe/) While satisfying that I could “make it go” and narrow the scope of where the problems were located, it limits the ability to reasonably test a new userland as any bug could be old kernel mismatch issues. Where 32 bit userland on 64 bit kernel is supposed to work.

Now you know why I’ve been somewhat quiet the last couple of days and not commenting quite so much on the political scene. I’ve been up to my eyeballs in bit shoveling ;-) Now that I’ve scratched this itch, I can come up for air and see what has happened in the world… other than World Cup and Trump Bashing… Or maybe that’s the empty set right now ;-)

Subscribe to feed

Advertisements

About E.M.Smith

A technical managerial sort interested in things from Stonehenge to computer science. My present "hot buttons' are the mythology of Climate Change and ancient metrology; but things change...
This entry was posted in Tech Bits and tagged , , , , . Bookmark the permalink.

28 Responses to My Magnificent Bastard System

  1. E.M.Smith says:

    Some links and refs:

    https://raspberrypi.stackexchange.com/questions/77693/enabling-armv8-on-raspberry-pi-3-b

    If you aren’t building a 64-bit kernel from source, you can download a working version from the original author of this article, here on GitHub. His image is based on Debian arm64, but if you intend to use this kernel with Raspbian make sure to copy in /boot/{kernel8.img,*.dtb,overlays,config.txt,cmdline.txt} and /lib/modules/ from the Pi64 image. Some Raspbian functionality will most likely break, so back up your existing 32-bit kernel configuration prior.

    A generally useful apt cheat sheet:

    https://www.cyberciti.biz/tips/linux-debian-package-management-cheat-sheet.html

    Where you can get the images for Devuan 2.0 (even if not as fast as the mirror, closer to the source – where I got my 2nd copy to compare SHA values)

    https://files.devuan.org/devuan_ascii/embedded/

    A detailed description of the boot sequence. Useful for when you want to swap some of it ;-) or to debug why it isn’t going, or just to find out what all those funny file names mean:

    https://raspberrypi.stackexchange.com/questions/10442/what-is-the-boot-sequence

    Just stuff you trip over when working on making a FrankenSystem ;-)

  2. E.M.Smith says:

    I observe a severe lack of comments on this thread and the immediately prior similar Pi Tech thread. If folks are un-interested in this kind of stuff, I can just not bother to post it.

    OTOH, if it is interesting, but you just don’t have anything to say about it, I can continue to post on my adventures in Tech Pi Land…

  3. H.R. says:

    E.M. – I read all your articles, but can’t contribute on the tech articles. I was taught Fortran as part of my engineering studies with a followup practical course in programming solutions to engineering problems. I also had a cognitive engineering class (design for the way humans think) that had us do some AI programming.

    My wife was the sysadmin for a large organization and now is only sort of tech geek. I was used to having the latest and greatest in hardware and software on the PC side of things. She ‘made’ me buy one of the first palmtop computers so she could play with it.

    Her stroke in 2008 knocked her out of the workforce and she has slowly fallen behind in the IT world. Just a few years out of the game is like a lifetime. I have pointed her to a few of your articles and point out links that might be of interest to her, links provided by you and the other regular geeks here.

    You enjoy keeping your hand in and I enjoy reading what you’re up to, but I just don’t have the chops to add to the discussion.

    H.R., reporting in to headquarters.

  4. p.g.sharrow says:

    Pretty hard to comment when you are in way over my head. ;-) I know just enough to follow your thorough descriptions of your explorations and discoveries.
    This bastardized Franken system is a stepping stone across this swamp of unknowns, education for you and entertainment for me. Sometimes the journey is as important as the destination…pg

  5. philjourdan says:

    Live and learn. I read them, but like PG, I am in over my head (but that is how you learn). Please do not take silence as not reading, just of trying to catch up!

  6. Steve C says:

    There seems to be a consensus forming here, and I’ll cheerfully add myself to it. We novices, who can just about make these things work, look on in awe as you display the system-fu needed to juggle half a dozen different file systems, FFS, into a neat pile in the Pi, and just for fun. We watch. Now and then, we learn. Thank you!

  7. I’ll also add my two-penn’orth in that not commenting here just means I haven’t anything relevant to add. I read them and at times bookmark stuff I’ll need to refer to later. These days I don’t drive a computer for a living, and it’s simply a tool that I want to work. I’ll only spend the effort if the available systems get sworn at too often. The one remaining Windoze box here gets sworn at pretty often, but does dual-boot. Finding software I want to use that doesn’t need Windoze can be a pain, but less pain than using Windoze.

  8. E.M.Smith says:

    OK, so interesting enough to keep doing it.

    I mostly do it so that 2 years from now when I need to do it again I don’t have to re-invent it all from scratch (including the errors…) but can just look up my crib notes. But I can use a paper binder for that just as well. Making such experiences a public posting lets others see, copy, or avoid as needed or desired ;-)

    I’d hope also that if someone else was having trouble with the Devuan 2.0 armhf install on a Pi M3, this would help them know they were not alone AND that there s a way past it…

    But I’m ok if it ends up just being “Ooooh, look at the high wire act… and he didn’t even fall”… yet…

    FWIW, the “fun” of it is finding just where your skill level runs out, and especially if it runs out somewhere just a bit beyond what you needed to get what you wanted. I wanted an all Devuan 2.0 installation. Not a Raspbian uplift. Not an Armbian converted. And with a working FireFox. Then ran into that brick wall of an official .img that would not run…

    But THE big thing about systems programmers and administrators that makes a “good one” is a strong desire to fix things and not have them screwed up; so much so that they just won’t stop until they find a way to “make it right”. Turns an “OMG I’m horribly stressed out the boss is on my case ’cause the server is down and I can’t make it go” into a: “Well this will be fun, time to get creative and solve a really hard puzzle!” Something of a “Never give up, never surrender!” thing.

    In the process you learn new stuff and add new tools to your utility belt. For me, now, that includes “mix and match” kernel / userland skills (I’d not done it before) and a much better understanding of the boot sequence and what all those funky files in /boot are for.. I guess I’m a sick puppy ’cause I found it fun to do that dive into the bit bank…

    I also wanted to get experience with xfs, so just used the same time to do that, too. “double your pleasure”… Just keep adding more “things at the same time” to just one short of the point where you lose track and screw things up ;-) Like a unicycle rider who adds singing to the act, then juggling , then juggling even more things, then singing while juggling lots of stuff while cycling backwards. Eventually you hit the crash and burn point, but just one short of that it feels Godlike. 8-)

    Unfortunately, now that this is working, that “hit” is over. Eventually the endorphins and serotonin wear off and I’ll need to find a new dragon to slay… But at least now I can use the Pi M3 to browse stuff and write postings again… The Mac is nice and all, but some things work better in an office chair with a real keyboard and monitor…

  9. E.M.Smith says:

    Well, I’ve set up for file system testing /comparison.

    I’ve taken a Toshiba 1 TB disk and made 14 or so partitions on it (!) with one each of btrfs, xfs, ext2, ext3, and ext4 that are all 8 GB each and all located near the same place on the spindle (i.e. there’s about 900 GB past them so only a few % variation in placement together).

    I’ve also made partitions for swap, /tmp, /var and /usr and copied those from the SD card to those partitions then rebooted to have them mount and be operational. (changed fstab entry and then rebooted into brave new world…)

    Here’s what the disk layout looks like now:

    chiefio@devuan:~$ df
    Filesystem     1K-blocks    Used Available Use% Mounted on
    /dev/root        1743136  213088   1439836  13% /
    devtmpfs          438328       0    438328   0% /dev
    tmpfs              88764    1664     87100   2% /run
    tmpfs               5120       4      5116   1% /run/lock
    tmpfs             387220       0    387220   0% /run/shm
    /dev/mmcblk0p1    130798   35754     95044  28% /boot
    /dev/mmcblk0p3   4062912 2854276    998924  75% /SD/arm64
    /dev/mmcblk0p5   3135488  603240   2532248  20% /SD/var
    /dev/mmcblk0p6   4184064 1694856   2489208  41% /SD/usr
    /dev/mmcblk0p7   2073600  586796   1486804  29% /SD/home/chiefio
    /dev/sda1        8378368   41392   8336976   1% /TN/xfs
    /dev/sda3        8388608   16704   7532352   1% /TN/btrfs
    /dev/sda5        8256952   18420   7819104   1% /TN/ext2
    /dev/sda6        8191416   36852   7718752   1% /TN/ext4
    /dev/sda7        8191416   18488   7753500   1% /TN/ext3
    /dev/sda8        8256952   18444   7819080   1% /tmp
    /dev/sda9        8191416   18488   7753500   1% /TN/root
    /dev/sda10       8191416  591352   7180636   8% /var
    /dev/sda11       8191416 1691584   6080404  22% /usr
    /dev/sda12       8191416 2867800   4904188  37% /arm64
    /dev/sda13      83845120  671484  83173636   1% /home/chiefio
    /dev/sda14     807535360  837712 806697648   1% /TN/store
    tmpfs             443800       0    443800   0% /sys/fs/cgroup
    tmpfs              88760       8     88752   1% /run/user/1000
    

    With duplicates of /var /usr /arm64 and /home/chiefio on both the SD chip and the real disk. (At some later time I’m going to boot the 64 bit os and copy the 32 bit / onto that TN/root partition and then try a direct boot to disk… I’ve done it before and it worked…)

    /TN/store is just the rest of the 800 GB of the disk for “whatever” use. Bit bag…

    The system already seems somewhat faster in FireFox. It will certainly have faster .cache files now (and it IS chatty to .cache…) along with any use of /tmp being faster. Oh, and loading any libraries / binaries from /usr too…

    During the copy from the SD chip to the disk partitions, the “D” waiting for disk symbol in the HTOP status display would light up red saying “stopped for disk activity”. It was ALWAYS on the tar doing the copy out from SD and never on the one restoring into the real disk. This tells me the SD card is limiting and the Real Disk(tm) is much faster.

    I know I could use “cp -r” to move things, but only on newer Unix / Linux machines (where newer is “from this millennium” ;-) Decades back I got used to using tar into a pipe to move things and I’m just comfortable that it doesn’t screw up ownership or permissions…

    cd /fromdir
    tar cf - . | (cd /todir; tar xvf -) 
    

    At this point it would be more work to NOT do that as it just flows out the fingers…

    It does have the feature that systems monitoring shows you what each part of the pipe (that | symbol) are doing. The first part “tar cf – .” just says to make a tape archive format bag of bits with “c” for create, into a file with “f” for file, then the marker that the file is NOT named in the file system, but is just a “pipe” into another program with “-” and put in it all the files in your current working directory “.” even those that are hidden. Then the “|” symbol says “Here’s your Pipe!” and sends that byte stream into a new subshell (created with “( )” ). In that subshell, it does a change of directory to the target umpacking place, and then uses “;” to go to the next command that is an extraction “x” of the tar archive format stream of bytes coming in from a file “f” that’s a pipe “-” and extract it right here please.

    Yes, that terse shell script bit said to do all that stuff listed in the big paragraph. Why *nix folks like it… It actually says a bit more via the default options to tar. Things like preserving modify times and owners and such, but that’s implied ;-)

    So now I’m running “mostly from disk” with only things that use the stuff in the root directory still coming from the SD card (and /boot at boot time…). So stuff use /root home directory and /bin or /sbin executables. There’s not a whole lot of them in userland though, so mostly this is going to be a “from real disk” experience.

    It has made things noticeably faster. I’ll have to think about ways to measure speed differences.

  10. E.M.Smith says:

    AND now I’m running with root from the hard disk too…

    Booted with cmdline.txt of:

    dwc_otg.fiq_fix_enable=2 console=ttyAMA0,115200 kgdboc=ttyAMA0,115200 console=tty1 root=/dev/mmcblk0p3 rootfstype=ext3 rootwait rootflags=noload net.ifnames=0

    Note: mmcblk0p3 where the arm64 build is located. This means NOT using the armhf build.

    Mounted mmcblk0p2 as /SD/armhf and copied it to the /TN/root partition on /dev/sda9

    changed the cmdline.txt entry to:

    dwc_otg.fiq_fix_enable=2 console=ttyAMA0,115200 kgdboc=ttyAMA0,115200 console=tty1 root=/dev/sda9 rootfstype=ext3 rootwait rootflags=noload net.ifnames=0

    Note: root=/dev/sda9 entry.

    and rebooted. Now running with /dev/sda9 as / (root) though due to them making it show up as /dev/root it is a bit hard to see in a df listing. But the size gives it away.

    chiefio@devuan:~$ df
    Filesystem     1K-blocks    Used Available Use% Mounted on
    /dev/root        8191416  228880   7543108   3% /
    devtmpfs          438328       0    438328   0% /dev
    tmpfs              88764    1656     87108   2% /run
    tmpfs               5120       4      5116   1% /run/lock
    tmpfs             387220       0    387220   0% /run/shm
    /dev/mmcblk0p1    130798   35754     95044  28% /boot
    /dev/mmcblk0p2   1743136  213088   1439836  13% /SD/armhf
    /dev/mmcblk0p3   4062912 2854404    998796  75% /SD/arm64
    /dev/mmcblk0p5   3135488  603240   2532248  20% /SD/var
    /dev/mmcblk0p6   4184064 1694856   2489208  41% /SD/usr
    /dev/mmcblk0p7   2073600  586796   1486804  29% /SD/home/chiefio
    /dev/sda1        8378368   41392   8336976   1% /TN/xfs
    /dev/sda3        8388608   16704   7532352   1% /TN/btrfs
    /dev/sda5        8256952   18420   7819104   1% /TN/ext2
    /dev/sda6        8191416   36852   7718752   1% /TN/ext4
    /dev/sda7        8191416   18488   7753500   1% /TN/ext3
    /dev/sda8        8256952   18444   7819080   1% /tmp
    /dev/sda10       8191416  591612   7180376   8% /var
    /dev/sda11       8191416 1691584   6080404  22% /usr
    /dev/sda12       8191416 2867800   4904188  37% /arm64
    /dev/sda13      83845120  677820  83167300   1% /home/chiefio
    /dev/sda14     807535360  837712 806697648   1% /TN/store
    tmpfs             443800       0    443800   0% /sys/fs/cgroup
    tmpfs              88760      12     88748   1% /run/user/1000
    

    That 8 GB root with 3% uses is kind of a dead giveaway ;-)

    So at this point, I use the SD card to boot an image from the hard disk, mount some other partitions for home directory /usr /var and such and everything goes to the USB hard disk.

    I’ve gone ahead an mounted the mini-SD card partitions (the mmcblk0p{n} stuff) so that I can, if I wish, change things in them. I’ve also got my experimental disk speed / use testing partitions (that /TN/xxxx stuff) and an on-disk copy of the 64 bit OS should I wish to try running it all from disk too (that /arm64 mount point partition).

    Yes, you could run it “all from disk” much simpler than this. Just one big / partition and not bother to mount the mmcblk* stuff at all, not have those disk file system test partitions, and not bother to split out /tmp /var /usr swap and /home to their own partitions. It would even be much faster at times too as all the empty space between partitions would be gone so seeks would be much shorter. (As it is, for historical reasons, the /dev/sda2 swap partition is at the bitter end of this disk. So it is a longest possible seek from the OS area to the swap area… That’s slow and not efficient. Better would be swap in partition 1, then the OS, then user space. )

    The Pi doesn’t swap much anyway, and when it does it tends to swap out idle stuff and then not need to page it back in ever. Also, were I to keep using this longer term, I’d add a second disk for swap and give it higher priority, only using this “remote” swap area if desperate. It’s my usual way of doing things. Swap on two spindles…

    As this is an experimental play space, I’m not that worried about optimal layout. At some future time, when filesystem play is over and when I’ve figured out if I like /usr on squashfs or not, then I’ll come back and optimize disk layout. Or just be “done” and scrub this system entirely ;-)

    As it is, with a second added swap and all the system partitions within about 32 GB of space out of a 1 TB disk, I think it is very much “good enough”.

    So that’s what I’ve got done today (so far).

    A multi-boot armhf/64 hybrid, running from either SD or disk at your choice, and a amr64 pristine, also running from SD or disk at your choice, with space to test / characterize file system types from either OS.

    Kinda fun ;-)

  11. jim2 says:

    I’m just a lowly programmer and don’t sit on the high perch occupied by the SysAdmin types, and I wouldn’t have been able to do what you did without a lot of google and pain.

    However, once you laid it out, I kind of get what you were doing, so I appreciate the explanations.

  12. E.M.Smith says:

    @Jim2:

    Conceptually it’s pretty simple, it’s the exact typing that gets confusing ;-)

    At boot time, the ARM looks on the first MSDOS partition for the bootoader program. It (couple of odd steps) eventually looks for cmdline.txt to tell any particulars. That includes what to load next (where to find the root file system) and what kind of file system it is. That’s where you can switch what partition on what device has a Linux that gets loaded…

    It can be loaded from a disk or from the mini-SD card, and any partition that can hold a Linux / partition.

    So I just loaded up 2 different versions (the 64 bit one and the hybrid 64 bit kernel / 32 bit userland) one on the SD card and one on “real disk”. That lets you point your boot at any of the 4 places.

    The other bit is just picking off some of the root file system name space and placing it on dedicated partitions. So mount some empty space as /TEMPSTUFF and copy, say, /var into it. Then you can unmount /TEMPSTUFF and mount it on top of /var. You can leave the old stuff there and just hide it with the mount, so if you want to boot without the disk there’s still a /var there; or you can erase the bits on the SD card making an empty /var and mount the copy from /TEMPSTUFF on /var like before (but with nothing hiding under it now).

    That’s pretty much it. I did make some of the partitions other file systems different types than the present default. Mostly as a learning exercise. I also carved out some small ( 8 GB) partitions of various file system types, again for play / learning. I used gparted for that and it’s your basic click and fill in the blanks GUI tool.

    That’s it. Not really all that much. Just need to spend “time in grade” trying it until you get familiar with the commands to do it…

  13. gallopingcamel says:

    Go Chiefio, go!
    I would love to try Rasperry Pi but don’t have a project that might need it. Probably a good thing as the magic you do is well above my pay grade.

  14. E.M.Smith says:

    @G.C.:

    Folks keep saying stuff like that, and I do recognize I’ve been doing this stuff for about 30+ years so I’m likely very biased / jaded about it; but:

    All of this stuff is simpler than writing programs or even making bread.

    The “hard bits” are mostly the arcane language of Linux / Unix systems admin and that it’s “new” stuff most folks have not thought about. The jargon of systems administration and the “complete but numbing” nature of reading the man page for something like tar. ( “man tar” gives you every possible thing you can do with it, but little guidance as to why you would do it)… Still, my constant companion is “man {whatever}” to find out what’s in a particular version and “apropos {foo}” to find out what man pages are apropos for that thing.

    But the reality is that rarely does a “thought” have to span more than a few lines of some scripting language. Then there’s the fact that we all had to learn it one command at a time. So often folks will just (now) do a quick web search on “how to copy file system” or “how to make partition” and read a few examples, then figure out which one works best on their flavor of *nix.

    Even Me.

    Why? Because this stuff changes from decade to decade and even distribution to distribution. (most recent disruption being that systemD changed huge chunks of how systems admin is done. My response being to just avoid it ;-) Then sometimes you have not used a thing for a decade and vaguely remember it, so need to reload the thing… That you don’t see me “loading up the skill and command set”, just see the product, is a bit of cheating on my part…

    An example:

    I used to make partitions “long hand” in Unix. UFS file system. Now I used gparted. I’d never even seen gparted prior to Raspbian on the Pi. ( I used other tools before that… on other *nix platforms. Commands like mkfs. I’d now have to look up how to use them again.) So at that day, when I got my first Pi, we were equally “skilled” at gparted. (There is a command line ‘parted’ that I’ve not used at all in decades and have completely forgotten, so we are equally “non-skilled” on it at the moment too ;-)

    So you pick one little thing. Say adding a file system to a new disk (or SD card). And you dig in and learn how to do that one thing. Maybe it takes an hour, or maybe 4. Now you have it.

    Then another day you learn how to copy a Linux .img file onto an SD card to install a new Linux. ( dd bs=1M if=linux_release of=/dev/sdaXY ) and another day you learn how to copy a file system from there, to a new partition.

    Now you have 90% of all the stuff I did above. Add a few percent for learning how to edit the cmdline.txt file (given above in about a paragraph…) and maybe one or two other trivial bits; and you too can do everything I did above. All up, maybe 2 days if being slow and inattentive about it…

    Caveat: IF starting from near zero computer experience so things like “partitions” and “partition tables” and “file systems” are all effectively Greek, well, then there’s some background understanding to learn… so give it a week of immersion in general computer guts ;-) and maybe another week of learning things like shell commands and how to write a script…

    So while I really enjoy the praise (who doesn’t?) the teacher in me wants to share the ability with others and encourage experimentation.

    The good thing is that each skill and each command is typically 100% stand alone and you don’t need to have a year of “something else” to use it. Each bit can be learned as a couple of lines of command text and one or two ideas about what it does. Nicely incremental. But you do need to take that first step… and then just one a day more…

  15. gallopingcamel says:

    @Chiefio,
    Thanks for that lengthy comment.
    You mention “gparted” which is something I use from time to time. It reminds me of “Partition Magic”, a program that cost me $50 in the bad old days before I became a Linux fan.

    I have a bunch of 16 GB microSDs and want to make a bootable system on one of them. There is enough room for “Mint” and my favorite Apps plus a few GB of files. The idea is to carry it around in my wallet with the intention of using it to boot up “Mint” on a foreign computer when I don’t have my laptop with me. I keep trying but can’t make the darn thing work. It has to be doable! Any suggestions?

    Recently I decided to spend a few weeks “travelling light” without my laptop. While I could handle emails and spreadsheets on my cellphone I missed the convenience of a full size display and keyboard…………..ended up using Windoze machines!

  16. E.M.Smith says:

    @G.C.:

    I’m assuming your target machine is a windows / intel PC on a desk somewhere…

    Essentially you just do a “live install” onto anything that plugs into a bootable port on the target machine. (Made harder now that folks are using EUFI to prevent changing boot / bios settings). “back when” you just pointed the BIOS at the USB / SD / CD / whatever and let it boot.

    While painfully slow, I defeated the “don’t boot anything else” protections on one corporate PC by installing a Solaris image in a VM on a 64 GB SD card… “There is always a way” meeting “It is a very bad idea to annoy the geek” ;-)

    So first off, what is your target hardware and what boot manager is it running? (Bios vs EUFI vs …) I think I’ve avoided buying any EUFI afflicted PCs so can’t model that case, but I can work up a “how to” for everything else…

    FWIW my “on the road lite” kit is a Samsung Galaxy Note 10.1 with charger. At 10 inches it’s better than a cell phone (i.e. usable) while not being too big for comfortable carry. The “virtual keyboard” sucks, but is livable. (It can use a bluetooth one if desired). Were I doing it over, I’d likely get an ACER Chromebook or similar. Only a little larger and at $179 or so last time I looked pretty cheap. It runs ChromeOS, but you can boot Linux on them…

  17. E.M.Smith says:

    My first file system test is done. A simple “copy from /dev/zero” that always provides bytes into a GB sized file and time it”.

    root@devuan:/TN/ext4# cat /usr/local/bin/gb
    time dd bs=1MB count=1000 if=/dev/zero of=$1 
    

    I named it “gb” for gigabyte. I also name the output file “gb”. So that means that invoking it to make a Gigabyte file would be “gb gb” ;-) The result looks like:

    root@devuan:/TN/ext4# ls -l gb
    -rw-r--r-- 1 root root 1000000000 Jun 19 15:39 gb
    

    Here’s the results:

    root@devuan:/TN/xfs# gb gb 
    1000+0 records in
    1000+0 records out
    1000000000 bytes (1.0 GB, 954 MiB) copied, 27.8861 s, 35.9 MB/s
    
    real	0m27.909s
    user	0m0.009s
    sys	0m6.600s
    
    root@devuan:/TN/xfs# cd /TN/btrfs
    root@devuan:/TN/btrfs# gb gb
    1000+0 records in
    1000+0 records out
    1000000000 bytes (1.0 GB, 954 MiB) copied, 32.9164 s, 30.4 MB/s
    
    real	0m33.226s
    user	0m0.021s
    sys	0m6.294s
    
    root@devuan:/TN/btrfs# cd /TN/ext2
    root@devuan:/TN/ext2# gb gb
    1000+0 records in
    1000+0 records out
    1000000000 bytes (1.0 GB, 954 MiB) copied, 31.914 s, 31.3 MB/s
    
    real	0m32.206s
    user	0m0.020s
    sys	0m15.587s
    
    
    root@devuan:/TN/ext2# cd /TN/ext3
    root@devuan:/TN/ext3# gb gb
    1000+0 records in
    1000+0 records out
    1000000000 bytes (1.0 GB, 954 MiB) copied, 32.6487 s, 30.6 MB/s
    
    real	0m32.694s
    user	0m0.021s
    sys	0m17.616s
    
    
    root@devuan:/TN/ext3# cd /TN/ext4
    root@devuan:/TN/ext4# gb gb
    1000+0 records in
    1000+0 records out
    1000000000 bytes (1.0 GB, 954 MiB) copied, 32.1307 s, 31.1 MB/s
    
    real	0m32.192s
    user	0m0.017s
    sys	0m11.901s
    root@devuan:/TN/ext4#
    

    So on creation of a single GB file, xfs is somewhat faster. None of them is dramatically different, but xfs looks to be enough faster to be worth it.

    Next I’m going to test creation / destruction of a lot of little files. Instead of exercising data blocks this exercises inode handling. Creation / destruction of metadata stuff.

  18. ossqss says:

    I am in a similar boat with EM on light travel. I use the Note 12.2 (and on my 4th non-removable replaced battery ;-)) and Office viewers instead of the Hancom suite (comes with the tablet), unless I need to edit. The BT keyboard case and BT mouse works quite well also. I take most notes with S-Note/pen and sync them to my Note phone so they are always available.

    OT, I got 4 likes last night on some of the items I posted. All from porn sites! I guess they like Trump.

    LOL, but a heads up also…..

  19. H.R. says:

    @osqss – If the likes were all from ‘Stormy Daniels’ then yeah, it was probably a Trump thing.
    😜

  20. gallopingcamel says:

    @Chiefio,
    I wanted to make a random Windoze machine run Mint from the microSD thinking it would behave like a “Live CD”. Probably slow as molasses but since I am also slow it won’t matter. That UEFI thing may explain some of my problems.

    Someone gave me a flash drive that installed Windows 7 on HP laptops (the only variant that worked with my OTDR program). It does not work any more and I can’t remember where it came from.

    I have a spare tablet that Verizon gave me. That might be easier for my clumsy fingers and poor eyesight than a cellphone.

    If the above don’t work out I will try the notebook approach.

  21. E.M.Smith says:

    @G.C.:

    The thing that is likely catching you is the move to EUFI boot that will NOT boot other than “approved” things. For that, you need to use the “slower than molasses” VM on an SD card that I did at (the place that will not be named by rhymes with house…)

    I’ll cook up a recipe and post it in a few days… Requires a command line boot of qemu…

  22. jim2 says:

    At one time I had Windoze 7 and Linux installed on my EUFI laptop. I believe gparted was part of the magic. Found explicit instructions on the IN, of course.

  23. E.M.Smith says:

    @Jim2

    If YOU are the sysadmin, anything is possible. if you are showing up at a “protected” site as a guest… More “Magic” is needed….

  24. E.M.Smith says:

    Did the test of file creation. A little script that touches 1000 file names, so creates 1000 inodes. No real significant difference between the various file systems.

    root@devuan:~# cat /usr/local/bin/files 
    
    for i in 0 1 2 3 4 5 6 7 8 9
    do
    for j in 0 1 2 3 4 5 6 7 8 9
    do
    for k in 0 1 2 3 4 5 6 7 8 9
    do
    #echo $i$j$k
    
    touch  file.$i$j$k
    
    done
    done
    done
    

    The results:

    The xfs and btrfs file systems are a tiny (1/10 second range) slower than the ext file systems (all done by the ext4 driver, just doing different variations). I’m not going to worry about 1/10 of a second on 1000 file creations… That’s 1/10000 second / file.

    root@devuan:/TN/xfs# pwd
    /TN/xfs
    
    root@devuan:/TN/xfs# time files
    
    real	0m5.695s
    user	0m1.154s
    sys	0m4.855s
    
    root@devuan:/TN/xfs# cd ../btrfs/
    root@devuan:/TN/btrfs# time files
    
    real	0m5.643s
    user	0m1.142s
    sys	0m4.780s
    
    root@devuan:/TN/btrfs# cd ../ext2
    root@devuan:/TN/ext2# time files
    
    real	0m5.508s
    user	0m1.136s
    sys	0m4.667s
    
    root@devuan:/TN/ext2# cd ../ext3
    root@devuan:/TN/ext3# time files
    
    real	0m5.586s
    user	0m1.135s
    sys	0m4.697s
    
    root@devuan:/TN/ext3# cd ../ext4
    root@devuan:/TN/ext4# time files
    
    real	0m5.656s
    user	0m1.159s
    sys	0m4.764s
    

    So in general, xfs is a little better on file data blocks but basically any of them is about the same overall. I’ll be using ext3 for most things and xfs for some user space file systems. As I get more comfortable, some systems file systems (like /var and /usr) may get moved to xfs. (As xfs is not installed by default this must be done carefully… to make sure you have xfs installed and running before moving the things onto an xfs file system… so no “build it on one with xfs” and just mounting it on another… without first assuring xfs is built there too.)

  25. E.M.Smith says:

    Things that send a sysadmin around the bend…

    So I have this nice DNS / Proxy server that caches my DNS lookups and web pages and all. Really cuts down on repeated traffic over the internet wire for things that don’t change and things that get “grounded” (like all the crap ads it blocks via DNS blocking).

    I’ve got this new install of Devuan. It’s a lite install in that I’ve not run my whole package of all the usual stuff in the script, just added bits as I needed them (it is just for testing after all…)

    So I’d not set it up with the full on local DNS service and such that I usually do. (So the localhost also caches DNS lookups and more) Just the regular “out of the box” user kind of build.

    BUT, I did do a configure of the ethernet interface to be a hard coded IP and pointed my DNS lookups at my DNS server in /etc/network/interfaces:

     auto eth0
     iface eth0 inet static
    	address 10.1.1.16
    	netmask 255.255.255.0
    	gateway 10.1.1.254
    	broadcast 10.1.1.255
    	dns-nameservers 192.168.15.253
    

    So I do a “nslookup yahoo.com” and find it doesn’t have nslookup installed. OK…

    apt-get install dnsutils 
    

    and I’m on my way… and do the nslookup again…

    It reports it is using the DNS services of 10.1.1.254. WT?

    Digging around, it seems that you can configure stuff in “interfaces” but it doesn’t actually use that DNS server unless you install another package. Really? How hard is it to just do what the line says and make that your DNS server? Sigh…

    https://packages.debian.org/jessie/resolvconf

    Resolvconf is a framework for keeping up to date the system’s information about name servers. It sets itself up as the intermediary between programs that supply this information (such as ifup and ifdown, DHCP clients, the PPP daemon and local name servers) and programs that use this information (such as DNS caches and resolver libraries).

    It’s one of those things I typically just install because I usually set up full DNS control on the local box, but never imagined that if it was NOT installed then a manually set IP address directive with a name server listed would have the name server line silently ignored.

    Oh Well… It’s working now and I once again have my ad blocking and caching services…

  26. jim2 says:

    When I do “nslookup google.com” I get this:

    nslookup google.com
    Server: 127.0.1.1
    Address: 127.0.1.1#53

    Does that mean I have a local DNS server or what?

  27. E.M.Smith says:

    It means your system is looking to itself for DNS services. What it does next is not stated. You would need to look into the configuration of your system (and even what OS it is…).

    Most likely you have some locally running DNS caching server, that looks upstream to other servers. I typically set up my Linux boxes that way (cuts down on pestering the Pi B+ that’s my network DNS server and does the actual looking outside the house for upstream DNS…)

  28. E.M.Smith says:

    Well, a comment on speed and comfort.

    I’ve spent much of today “polishing” my Devuan 2.0 boot system. I’ve gotten rid of the various file system testing partitions (having decided on ext3 for anything that needs to boot as xfs might not be there for the boot loaders tiny brains) and xfs for “general use after booted and OS configured with added xfs driver”. All efficiencies and performance are ‘about the same’ with the exception that xfs is a bit fast than ext(whatever) and with the understanding that at boot time a wrong date stamp (i.e. booting an old system that doesn’t know what time it is yet and thinks it is last month) causes ext(2-3) to do a full fsck while xfs just gets mounted.

    (No comments from the peanut gallery about “Is xfs cute?” OK?)

    And yes, fsck is often pronounced “F-SUCK”…

    Right now I’m posting this from a Devuan 1.0 system. It is faster and more comfortable than the Devuan 2.0 build. No, I don’t know why.

    I’ve done everything I can to make Devuan 2.0 fast and comfortable (shut up! I heard you!) but it is just a bit of a slow cow. I have it running from disk. I’ve got all file systems localized to the front 8 GB of a 1 TB disk. I’ve tried 64 Bit and Hybrid ( 64 bit kernel and 32 bit userland – I could not get all 32 bit to run).

    It’s just slower.

    In both cases I’m using Firefox on the same R. Pi M3. This one is fast and comfortable. The Hybrid or the 64 Bit is slow to painful but barely workable.

    What can I say. “It is what it is. -E.M.Smith” (shamefully stolen from someone who I”ve forgotten long long ago long ago…)

    I’ll continue to hope that Devuan 2.0 can pick up the pace as it matures. I suspect that the kernel fixes to fix Specter and Meltdown have had their bite and it is a big one.

    Time will tell…

    So, what to do.

    Stay back level on a Devuan 1.0 that’s fast and comfortable?

    Embrace the security of a slow plodding Devuan 2.0 that fixes stupid errors by HW guys?

    Spend some time in each as needs dictate?

    Sigh.

    OK, for posting stuff I’m staying back level as it works well and is fast. For general systems maintenance things (moving GB around disk to disk in my own shop) I’ll use the new 2.0 release on the newer more secure kernel (yes, I know that from a security POV hat’s exactly wrong… read on…) and for external not-my-site things I’ll use an isolated dedicated platform. For now, the Mac.

    Usually, over time, once “Hey, it WORKS!!” is out of the way, folks raise the speed and efficiency. I hope that continues to be the case.

Anything to say?

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.