Cluster distcc First Fire

First distcc build on Devuan Cluster Screen Shot

First distcc build on Devuan Cluster Screen Shot

Well, it is either a partial success or a full success… I’ve gotten my first compilation to run with distcc spread over the new cluster based on Devuan.

You can click on the image for a bigger version.

In the picture, the top left panel is a terminal window with a “make” of the Pi Linux Kernel being run on my Pi Model 3. The bottom left tiny window is where I ran “scrot” to make the SCreen shOT. The three panels on the right are running “top”. It shows what processes are running and some other operational aspects like memory used and swap and such.

In order, top to bottom, are the “top” reports for the Pi-M3 workstation showing one C compile running (CC1), under it the panel for “Headless 1” showing several CC1 instances running, then under that “Headless2” showing none running.

Now I’m not sure why none are running on “Headless 2”. It ought to have an even share of the load with Headless 1, but I might have a tuning parameter wrong or it might not be participating in the pool. That debugging is for tonight (now that I have a working test case).

The “biggie” for me is that I’m posting this on the Pi Model 3, with little notice of the build happening, since it is only running the occasional one compile in one cpu, while most of the work is farmed out to that other Pi Model 2 board. Nice, that.

So time to celebrate a tiny bit! Yay!!!

I “lost” about 3 hours today trying to find something to compile as a test case. I found free and open source compiles for a fancy BASIC, that was written in itself… Sigh. For a nice C, that wasn’t willing to try compiling on ARM. For g95 Fortran that I’ve compiled before, that ceased development in about 2008 so doesn’t know what an ARMv7 target is… and more junk.

Finally I had a bit of clue that maybe, just maybe I ought to simply build the Pi kernel itself since by definition it will have no such issues. Ta Da! How To here:

In Conclusion

Don’t expect to see any such thing for FORTRAN compiles. It doesn’t use distcc. OTOH, compiling Model II on the Pi Model 3 takes about 3 minutes, so who cares… the models are NOT big chunks of code. (Especially not compared with Linux Kernels, tools, compilers, and most large commercial codes. Heck, even a browser is much much larger.)

So what good is it?

Well, for one, it is a demonstration case that the cluster works. It also will really help with things like “rolling my own” distribution and building one from sources. (so that week to build BSD can become a day, or less…)

It is also a ‘shake down’ for the cluster. If it can compile a working linux, it can run a compute intensive model without heat or other modes of failure.

So, with those steps out of the way, I’m on to the next bits.

1) Get “Headless 2” to participate in the compile party.

2) Build Model E and test the MPI distributed execution.

3) Run Model II and see how long it takes. Maybe try adding some MPI bits to it.

4) Add the Banana Pi 3rd headless node to the cluster and see if it “plays well with others”.

5) Finally get around to building that UPS I wanted…

Power failed today for about 1/2 hour in the storm. The Pi’s did a great job of rebooting on their own, but I’d rather not have that happen in the middle of a Linux build. I’ve got a kW UPS with a dead gel battery in it, and I’ve got a BIG Mercedes sized battery that could stand to live on a trickle charger… soo… Battery to box. Clean up UPS and remove battery. Add jumpers. Viola!… (Pronounce Vi-Ola for effect ;-) NOT an error in Voilà)

With that, I’m taking a dinner break and a movie break and a just sit and glow a bit break ;-)

Happy Hacking!

Subscribe to feed

Posted in Tech Bits | Tagged , , , , | 15 Comments

TT – 2 Weeks (and counting the days)

The Trump Transition “TT” at minus 2 weeks.

Just 2 more weeks in January. Sporadically I’ll put up additional TT posts as any one gets too full to load quickly, or as some event causes me to desire to put a head post up on the topic.

Prior threads in the chain are:

We skipped 4 due to sloth on my part…

So Trump had his Spy-vs-Spy briefing, and got it only slightly later than the Press got a copy… one wonders why THAT leak isn’t an issue…

The Russians Did It! Is losing currency in the USA due to the vapid nature of it. Golly, the Russians let us see the truth? That’s what you got? My assessment of it is not that far from the actual level of support from the Intelligence Community (Hannity did a great rip on it tonight). They differ mostly in that the Official Report (after saying we have evidence the Russians are up to something, a lot, but can’t really finger them) proceeds to finger them in the conclusions. The Looney Left News tries to spin that as proof, while the actual analysts in the TLAs know their bosses are political.

BBC tried to keep the spin up with saying Trump was in a food fight with the Agencies and was threatening to reorganize them. From my POV, I’m just wondering why in hell we need 17 Agencies (especially if they all agree… one would do…) doing the same job. Imagine yourself in Moscow at an anti-USA meeting and you find out of the 25 people there, 17 are US Intelligence Agencies… That only leaves 8 for The UK, Germany, Ukraine, Israel, Saudi Arabia, Iran, China, Japan, Poland, and Brazil… somebody is going to be left out… oh, yeah, any actual Russians…

Yet D.W. (German news) is still lapping it up. Tonight they even claimed the next Russian Target was Angela Merkel. Never mind her incredibly stupid open door policy for immigration that has resulted in death, destruction, and rampant groping of women. No, if she loses, it will be because of Russian induced Nationalism…

Even Biden has realized Trump is President. Ending the attempt at a food-fight by the Dimocrats in his Democrat party with “It’s Over” and slams the gavel. Certifying the vote.

What will the next 2 weeks bring? I suspect more bratty food fights from the Left who just can’t accept that We The Deplorables had a good close look at their candidate and decided we didn’t want to spend another 4 years looking at that; and maybe shutting down industry with “trade deals” and mining due to non-existent “Global Warming” would turn voters against you. (Oh, do note the rampant cold right now… Sierra Nevada expecting about 4 to 6 feet of “global warming” at the ski resorts…)

About all they’ve got left was MSNBC (or maybe CNN or maybe both, hard to tell them apart) moaning Trump was reneging on his “Mexico Pays for the fence” promise since he was going to build it first with USA sourced funds then collect payments later… Have they never heard of a mortgage?… Mortgage the few $Billion of funds transferred to Mexico every year by creaming off a penny on the dollar and you get your money back pronto. I’m sure the illegals working here and sending it home won’t complain… (or maybe Trump can set up “complain and get a free ticket home” stations for them in all major cities.)

May the end of the Obama “rhymes with bucket” list come soon… what he’s done so far trying to piss in the punch bowl of the inauguration is already horridly too much and very wrong. But hey, why change his style after 8 years of it…

With that, let the conversation roll on…

Subscribe to feed

Posted in Political Current Events | Tagged , , | 117 Comments

Devuan and Climate Cluster Milestone

Just a short note on some hardware status.

As of this moment, I’ve got 2 x Pi Model 2 boards, in the dogbone stack / case, running on a mostly built as I like it Devuan Jessie. My desktop at the moment is a similarly built Devuan, but with the hardware being my Pi Model 3. That’s a total of 12 cores available for use in a cluster formation.

They all have distcc installed (though not yet configured) so I can now do compiles of code spread over many cores. But I’ve set it up before, so configuration won’t take long.

All of them were made from a NOOBS base installation of Raspbian, then upgraded to Devuan.

I then ran the script that I’m including below. Note that many bits are commented out. Some of them I just didn’t want on these systems (like a VNC remote terminal). You can customise this list to suit your needs. For example, I’ve got gfortran and distcc stuff in it. If you are not going to be doing big compiles or FORTRAN on your system, you don’t need them.

Here’s the “disk” (chip) usage:

Filesystem     1K-blocks     Used Available Use% Mounted on
/dev/root       15951288 10406200   4711760  69% /
devtmpfs          469532        0    469532   0% /dev
tmpfs              94776      416     94360   1% /run
tmpfs               5120        4      5116   1% /run/lock
tmpfs             189540    47624    141916  26% /run/shm
/dev/mmcblk0p6     67434    20698     46736  31% /boot
/dev/mmcblk0p9  13157084    34988  12430692   1% /Climate
/dev/mmcblk0p8     67434        0     67434   0% /boot0
/dev/mmcblk0p5     30701     1563     26845   6% /SETTINGS
cgroup                12        0        12   0% /sys/fs/cgroup
cgmfs                100        0       100   0% /run/cgmanager/fs
tmpfs              94776        0     94776   0% /run/user/1000

Note that I have this on a 32 GB “chip” (mini-SD card) and have it set up with a /Climate directory where I’m presently working on GISS GSM Model II. Don’t let the size of “/” worry you. I had to dump a 6 GB USB thumb drive into that for a different use… The actual install takes about 6 GB and it fits on an 8GB ‘chip’ if you don’t make a /Climate directory. (One of the Headless nodes is using an 8, the other a 16, this workstation node has a 32 since this is where source and compiled bits will live.)

There is still a fair amount of configuration to do since I made these from the ground up. Things like turning on distcc, enabling ssh so I can remote log in (just found that out when I brought them up headless…) making sure they are booting with fixed IP (right now, one is, the other isn’t) and have a set name (“Headless1” and “Headless2” are now set). That kind of stuff.

One big bit I need to do: These by default boot up live with a GUI open as user Pi. I want to have a login prompt that asks for a password AND my “usual and customary” login name and home directory… So that is yet to do.

I tried to do a Linpack performance benchmark, but it would reach 396% of CPU and hang. That was on a Debian SystemD chip, so I hope it doesn’t hang on these. (It was after that I ‘moved on’ to doing the cluster build).

The boards are just plugged into the ethernet ports on my interior WiFi router for now. As the Pi is only 10 / 100 Mb Ethernet, that Netgear router is plenty fast for them. When I go over 4 (IF I go over 4) I’ll need to fire up my 8 port Netgear Switch…

Moving Forward

I’m now in a position to try running the Model II GCM code, and then I can start looking at Model E (that has distributed MPI already coded into it). If needed, I can put similar changes into Model II and make it distributed execution (though I’d rather not just yet).

Most importantly, I can make some needed tests of MPI function and efficiency. OpenMP (thread parallel) was dismal in an earlier test on Debian, it actually ran slower! Hopefully MPI (MPICH2 – core parallel) will actually have speed increases…

Essentially, I have my Toy Cluster built and ready to play with, test, and characterize. I think it will be doing “interesting things” by Sunday. (Spouse has spoken for Saturday as Macintosh Fixing Day… new disk, load data, etc…)

So far there is only one “bug” I’ve seen in this release of Devuan. When you choose to logout, it gives a panel with choices of “shutdown” and “exit to prompt” or some such. Shutdown doesn’t work, so you must chose the exit to terminal line prompt. That then doesn’t give you a terminal line, but another panel where shutdown does work… Petty annoyance, so might take a while for Devuan to fix on the Pi.

Overall, I’m quite happy with it, and I’m very happy to be done with the long search for a way out of systemD hell.

The Build Script

Here’s my “build script” for the Devuan installation. As is my custom, I make one of these for any build I expect to do more than once. It references earlier postings on the earlier steps (doing a Debian installation) and points at the reference for the Debuan upgrade that I found most useful (when making my first Devuan chip in the article here )

The Script:

root@raspberrypi:/media/pi/PINK_ext4/Pi_Config_Files# cat BuildDevuan 
echo " "
echo "Do the NOOBS install: "
echo " "
echo " "
echo " "
echo "Do the Raspbian installation.  Then copy this script from an external SD"
echo "card or USB drive into your working directory (home directory or /media/data"
echo "after all the usual and customary NOOBS setup questions are answered and done."
echo " "
echo "Make sure you select the USA keyboard if in the USA, default is UK!"
# In general, I'm encapsulating what all I did in these two postings as a script:
# If you didn't already change the password while running NOOBS,
# When done, log in as 'pi' password 'raspberry'.  Change the password.
# passwd
# and respond with the new one when prompted.

echo ""
echo "Do the upgrade / change to Devuan from this posting:"
echo ""
echo ""
echo " "
echo "It basically consists of changing /etc/apt/source.list to have:" 
echo ""
echo "deb jessie main"
echo ""
echo "as the only active line. Then do:"
echo "apt-get update"
echo "ignore the error message about keyrings and signatures, you don't have it yet."
echo "apt-get install devuan-keyring"
echo "Now you have the keyring... and can do another apt-get update if you want one"
echo "with no error messages, but it doesn't matter."
echo ""
echo "apt-get install sysvinit-core"
echo "apt-get install base-files"
echo "That last one may say you are already up to date."
echo "The next one will kill Berry Boot images with an attempted kernel swap."
echo "apt-get dist-upgrade"
echo "apt-get autoremove --purge"
echo "# apt-get autoclean"
echo ""
echo "Now reboot to start running Devuan properly.  If you don't have xfce, add it"
echo ""
echo "apt-get install xfce4"
echo "and startxfce runs it." 
echo ""
echo "Also, to change the name of your machine, edit /etc/hostname and make it"
echo "what you like.  "
echo "Here, I'm going to just set mine by brute force write to the file."
echo " "
echo "echo 'Devuan_armhf' > /etc/hostname "
echo " "

echo 'Devuan' > /etc/hostname 

echo " "
echo "Next, do the 'usual' update upgrade that brings you up to the present"
echo "repository status (need a network connection from here on out)"
echo " "
echo "You can either put 'sudo' in front of each of these commands, or just "
echo "'become root' which is what I usually do."
echo " "
echo "sudo bash"
echo " "
echo "then run this script with ./BuildIt (assuming you didn't change the name"
echo "and that you are 'in' the directory where it is located.)"
echo " "
echo "apt-get update"
echo "apt-get upgrade"
echo " "

#apt-get update
#apt-get upgrade

echo " "
echo "Start doing useful operational 'packages'. "
echo " "

# This gets the useful tools like "nslookup" for looking at Domain Names

echo " "
echo apt-get install dnsutils
echo " "

#apt-get install dnsutils

echo " "
echo " VNC is a nice way to get a remote desktop.  It takes some configuring later."
echo " "

echo " " 
echo apt-get install tightvncserver
echo " "

#apt-get install tightvncserver

echo " "
echo "I like wicd for an easier way to manage wireless devices and networks."
echo " "

echo " " 
echo apt-get install wicd
echo " "

#apt-get install wicd

echo " "
echo "Scrot is a tool for taking screen shots by saying 'scrot' in a terminal"
echo " "

echo " " 
echo apt-get install scrot
echo " "

#apt-get install scrot

echo "Normally I would install 'build-essential' to get things like C compiler"
echo "and some language tools, but they were already installed on the R.PiM2."
echo "Now I add gfortran and distcc"

#apt-get install build-essential

apt-get install gfortran distcc distcc-pump dmucs ccache doc-base gfortran-4.9-doc libgfortran3-dbg

echo " "
echo "Some 'user land' useful things like browser options and Office / Mail tools."
echo " "
echo "Chromium is the 'chrome' browser from Google but in Linux land"
echo " "
echo "    CHROMIUM is installed by default now"

echo " " 
echo apt-get install chromium
echo " "

#apt-get install chromium

# IceApe is a "more free" version of IceWeasel that is a "more free" version of
# Firefox that is a rebranded Mozilla that is...   IceDove is the matching
# Thunderbird replacement minus the trademarks, non-free bits, etc.

echo " "
echo "Doing IceApe browser and IceDove mail reader"
echo " "

echo " " 
echo apt-get install iceweasel
echo apt-get install icedove
echo " "

apt-get install iceweasel
apt-get install icedove

echo " "
echo "GIMP is the photo editor ( 'photoshop Free'...) "
echo " "

echo " " 
echo apt-get install gimp
echo " "

apt-get install gimp

echo " "
echo "Don't forget Libreoffice - Microsoft?  We don't need no steenking MicroSoft..." 
echo " "

echo " " 
#echo apt-get install libreoffice
echo "Already installed by default"
echo " "

#apt-get install libreoffice

# I tried "arora" and got error messages and "xbmc" was not working for sound
# so I'm not installing those again until it's clear they work.  Arora is a
# browser (so who needs "yet another browser" with Epiphany in by default and
# with both IceApe and Chromium installed?...) and I'll likely make a dedicated
# SD card for the media center option as there are 2 Pi Model 2 version up
# and I won't need to screw around with Debian issues...
#apt-get install arora
#apt-get install xbmc

# As I also wanted one of these to be a bittorrent server, I sometimes add
# the "transmission" bittorent code.

echo " "
echo "Adding the 'transmission' bit torrent server"
echo " "

echo " " 
echo apt-get install transmission
echo " "

apt-get install transmission

echo " "
echo "To get NTFS disks (like USB or an NTSB formatted SD card in adapter) to "
echo "work 'read write' instead of just 'read only', you need ntfs-3g"
echo " "

echo " " 
echo apt-get install ntfs-3g
echo " "

apt-get install ntfs-3g

# In Theory, this installed 2 VNC "viewers" so the R.Pi could use VNC to 
# get to other machines.  In practice, I found that one of them locked up
# my console when launched against my own machine as target (might be a 
# PIBKAC problem - Problem Is Between Keyboard And Chair - as the R.Pi
# isn't really expecting to drive 2 video sessions at once (the real one
# and the VNC one inside the real one...) so maybe all is fine and I just
# need to RTFM (Read The, er, "Friendly" Manual) before using software...

echo " "
echo "Some VNC Viewers for being the client instead of the server"
echo "I've not used either of these yet so have no clue about them in practice"
echo " "

echo " " 
echo apt-get install xtightvncviewer
echo apt-get install ssvnc
echo " "

#apt-get install xtightvncviewer
#apt-get install ssvnc

echo " "
echo "Want an NFS (Network File System) server so you can share disks with" 
echo "your internal network?  This will install the code, then you get to" 
echo "configure things like /etc/exports"
echo " "

echo " " 
echo apt-get install nfs-kernel-server
echo " "

apt-get install nfs-kernel-server

# prior to first use.  Or reboot.

# In your /etc/exports file, put something like:

# /etc/exports: the access control list for filesystems which may be exported
#		to NFS clients.  See exports(5).
# Example for NFSv2 and NFSv3:
# /srv/homes       hostname1(rw,sync,no_subtree_check) hostname2(ro,sync,no_subtree_check)
# Example for NFSv4:
# /srv/nfs4        gss/krb5i(rw,sync,fsid=0,crossmnt,no_subtree_check)
# /srv/nfs4/homes  gss/krb5i(rw,sync,no_subtree_check)

# /YourFileSystem  *(rw,sync,fsid=0,no_root_squash)
# But without the # in front of YourFileSystem... and with your file system...

echo " "
echo "IF you chose that 'add a 512 MB partition option' at build time"
echo "This adds it to the /etc/exports file so it is NFS mountable elsewhere"
echo " "
echo "echo '/media/data   *(rw,sync,fsid=0,no_root_squash,no_subtree_check)' >> /etc/exports"
echo " "

#echo "/media/data   *(rw,sync,fsid=0,no_root_squash,no_subtree_check)" >> /etc/exports

# Remember to do a 

echo " "
echo "Restarting the appropriate services so NFS will work"
echo " "
echo " " 
echo service rpcbind restart
echo service nfs-kernel-server restart
echo " "

service rpcbind restart
service nfs-kernel-server restart

# I also made my box a static IP number as it's a server.  You will need to
# make this your own server name and IP numbers.
# Here's my /etc/network/interfaces file with leading # to make it comments.
# I will make this a "dump these lines in to replace" in my running version.

echo " "
echo "Remember to make your /etc/network/interfaces file have a static IP#"
echo "If you are going to be using PXE boot and such"
echo " "

#auto lo
#iface lo inet loopback

#auto eth0
#allow-hotplug eth0
#iface eth0 inet static
#dns-domain chiefio.home
#auto wlan0
#allow-hotplug wlan0
#iface wlan0 inet manual
#wpa-conf /etc/wpa_supplicant/wpa_supplicant.conf
#auto wlan1
#allow-hotplug wlan1
#iface wlan1 inet manual
#wpa-conf /etc/wpa_supplicant/wpa_supplicant.conf

# Don't forget to do a
# ifdown eth0
# wait a minute for it to quiet down
# ifup eth0

# As I want this to be a DNS server, DHCP server, and PXE server (uses a 
# tftp or "Trivial File Transfer Protocol" server, all of those can come in
# one package with dnsmasq.

echo " "
echo "Installing a light weight but effective DNS, DHCP and TFTP service"
echo " "

echo " " 
echo apt-get install dnsmasq 
echo " "

apt-get install dnsmasq 

echo " "
echo "Yes, it takes configuring.  See the file at"
echo " /etc/dnsmasq.conf"
echo " "

# Then I installed the Apache web server :


echo " " 
echo "Instlling the Apache Web Servier and related stuff"
echo " "

echo " " 
echo apt-get install apache2 apache2-utils apache2-doc
echo " "

apt-get install apache2 apache2-utils apache2-doc

# and yes, it takes some configuring and even web page building.
# See files in places like /etc/apache2/sites-available and more.

echo " " 
echo apt-get install libapache2-mod-php5 php5 php-pear php5-xcache
echo " "

apt-get install libapache2-mod-php5 php5 php-pear php5-xcache

echo " " 
echo apt-get install php5-mysql
echo " "

apt-get install php5-mysql

echo " " 
echo apt-get install mysql-server mysql-client
echo " "

apt-get install mysql-server mysql-client

# These needed to be added for Transmission and for file systems

apt-get install transmission-daemon
apt-get install btrfs-tools xfsprogs gparted f2fs-tools unionfs-fuse
apt-get install hfsplus hfsutils
apt-get install squashfs-tools aufs-tools

echo " "
echo "And that's the end of my present install build process."
echo " "
# There are several files to edit and configure.  Eventually I'll add a 
# "here script" to dump them from this script to where they belong, or 
# I'll just save a copy and have a 'save / restore' copy process.
# Once I get everything configured ;-)

Folks may notice this looks mostly the same as the Debian build script. Some things commented out. Some added. A few, like iceape, removed since there is not target for it to build anymore.

For anyone looking to get a working Devuan on a Pi, with most of the goodies needed to do interesting things, this ought to get you there.

FWIW, it took me about 5 hours from start to end. (Part of the duration will depend on network speed… especially that part where the apt-get dist-upgrade says to go get a cup of coffee…) I made copies of the chip at the NOOBS installed stage, at the Devuan raw upgrade, and after my scripted added packages. (dd bs=10M if=/dev/sd{your drive letter} of=/where/you/save/it) using the 8 GB chip for the basic build, so less to store. Then later did a similar dd from the final saved image onto the 16 GB chip. That leaves about 8 GB not allocated, so with gparted you can make that /Climate partition…

During much of that time you can be doing other things. Do realize that during the Devuan upgrade it does prompt a couple of times, and at one point tosses you into a log file reader. I think I did :q to get out of it by vi instinct, and who knows what the proper exit command is… so it isn’t entirely ‘fire and forget’, but more like “fire and check back”.

Now, with my saved chip images, I can skip all that in the future and dd it onto any new chips I want to make, then modify the config.

With that, it’s time for a break! Happy Hacking!

Subscribe to feed

Posted in AGW and GIStemp Issues, GCM, Tech Bits | Tagged , , , , , | 12 Comments

Hey P.G.!

Hey, P.G.!

It worked… Yes, I finally got around to checking my email…

pg sharrow sent you $20.00 USD
Note from pg sharrow:

quote this is a test. . . pg


Also, in checking, I found I have enough in there to buy about 5 more R.Pi boards and support stuff (power supply, etc.) in total. That means that whenever I get the GCM Model codes running, I can increase the size of the compute farm fairly quickly, if needed.

(Thanks to all who have contributed, even if I didn’t read the email yet!)

I have ordered an Orange Pi board, at $16 each plus power supply makes it $25, and Orange Pi make the cheaper $10 one (that wasn’t in stock at Amazon… so I’m not getting it right now…). I’m going to ‘bite the Chinese bullet’ and evaluate it for function and security… It has the same V7 instruction set as the Pi M2, so ought to interoperate in a cluster well. So you have directly funded both an investigation of the suitability of the Orange Pi family of boards, and expanded my compute cluster.


This is the gizmo:

Orange Pi board

Orange Pi board

IFF the test is successful, then those Orange Pi Zero boards at $10 each (all up shipped) as headless compute nodes make a relatively high performance cluster dirt cheap. They are a bit light on memory, so it will depend a little on how benchmarks run on these vs Raspberry Pi vs Intel boards. Then again, headless takes a lot less memory.

It may well be that the “computes / $” are best with a couple of multi-core Intel boards from Fry’s and a bit of tinkering. I’ve done the DIY computer build thing many times, and building up a headless board is pretty easy… I have a half dozen Intel based old boxes that could be clustered, but the reality is that the 486 and Pentium I class chips in them suck enough power per compute that it would be cheaper to run a Pi cluster of equal computes (The Pi Model 2 is more computes…) So it would likely only be new Intel chips that would have enough computes / Watt to make it valuable, and I only have 2 of them runnable right now. (Both having disk issues, but that doesn’t matter in a headless PXE boot compute node).

But I’m getting ahead of myself…

First make the model run.

Second function test the port to Raspberry Pi, Orange Pi, Intel; and how well things parallelize.

Third benchmark alternatives.

Fourth make hardware recommendation for what would work best per $

Fifth buy and build…

Hey C.D. Quarles!

I got the input files for the Model! LOADS of thanks!!

I can now actually give it a spin and see what happens.

They may, or may not, need tweaking, but even if they do, that’s a heck of a lot less work than creating from scratch. Generally, it looks like “model experiments” consist of tweaking the input files anyway, so likely “Many such journeys are possible”… (TNG – Guardian of Forever).

So thanks to all, and special h/t to C.D.Quarles and P.G.!

Subscribe to feed

Posted in Human Interest, Tech Bits | Tagged , , , , | 9 Comments

Brats and Prats of the World Unite! – Dems Tantrum Attack

Well, we have enough evidence now to know how the Democrats intend to operate in the Trump years.

The intent is to be brats and prats and throw a tantrum and refuse and scream. I.e. all the best tools of a 3 year old.

For any non-native English speakers, the word “prats” might not be familiar, so a definition:

prat (præt)
slang an incompetent or ineffectual person: often used as a term of abuse

n. Slang.
the buttocks.

Noun 1. prat – the fleshy part of the human body that you sit onprat – the fleshy part of the human body that you sit on; “he deserves a good kick in the butt”; “are you going to sit on your fanny and do nothing?”
arse, ass, behind, buns, buttocks, hind end, hindquarters, keister, nates, posterior, derriere, fanny, rear end, tooshie, tush, seat, fundament, backside, bottom, rump, stern, tail end, tail, rear, bum, can, butt
Based on WordNet 3.0, Farlex clipart collection. © 2003-2012 Princeton University, Farlex Inc.

Now while that last one limits it to the human body, IMHO nothing in the common usage of “prat” prevents it from poetic use as “horses ass”…

I find it especially useful when “He is an ass” would be unacceptable or blocked…

One Example

The news has been full of stories of such planning for obstruction. Fox was reporting that the State Of California has offered up Eric Holder as their point man in being The Biggest Prats In The Nation!! (i.e. obstructionists in chief… OOTUS? Obstructionist Of The US?)

California hires former AG Holder to challenge Trump

California lawmakers already are preparing for a legal brawl with the Donald Trump administration – and they’ve got President Obama’s former top attorney in their corner.

Top state Democratic lawmakers announced Wednesday that former Attorney General Eric Holder has been tapped as outside counsel to advise the Legislature on potential challenges with the Trump government. He will lead a team from the Covington & Burling law firm, where he’s been working since leaving the Obama administration in 2015.

That one is well worth reading, but the one I want to mention is this one:

It has the wonderful lead-in picture of THE Reason the Democrats have gone rushing to the basement. It has Obama in the lead, Pelosi and Schumer sucking up behind, and all three of them with Idiot Grins on their faces.

Obama, Schumer, Pelosi with idiot grins

From Linked Buzzfeed Article

Prats & Brats On Parade at it’s finest…

I’ve bolded some bits in the quote:

Democrats Say They Won’t Help Republicans Craft Obamacare Replacement

Democratic leaders came out of a meeting with President Barack Obama Wednesday saying they will refuse to help Republicans if they repeal the Affordable Care Act without a replacement plan.

posted on Jan. 4, 2017, at 6:10 p.m.

Paul McLeod
BuzzFeed News Reporter

WASHINGTON — Congressional Democrats came out of a strategy meeting with President Barack Obama vowing to refuse to work with Republicans if they repeal Obamacare without a replacement plan.

Republicans have frequently promised to “repeal and replace” the Affordable Care Act. But while they are vowing to move rapidly on repeal, there are no details on a consensus replacement plan yet.

Obama met with Democrats on Capitol Hill Wednesday morning to chart out a plan to save Obamacare. They emerged from the meeting announcing they will not work with Republicans to craft that replacement plan if Republicans insist on forging ahead on repeal.

“We are telling Republicans, if you are repealing, show us what you’ll replace it with first and then we’ll look at what you have and see what we can do,” said Senate Minority Leader Chuck Schumer.

“They’re repealing, we’re not. It’s their obligation to come up with a replacement first and I think we have unanimity within our Democratic caucus on that position.”

Though Republicans are about to control both Congress and the White House, Democrats say they can still rely on the court of public opinion and Republicans are going to quickly find themselves in trouble without a replacement for the health care law.

Their whole strategy comes down to being a Refusenik and fomenting obstruction. This is going to impress the Joe Sixpack and Jane Working Girl of the USA exactly how?

It IS in line with the Color Revolution approach (more on that in some other posting), and it IS in line with the Socialist Alynsky Method, and it IS in line with Community Organizing Propaganda (just what to expect from the Community Organizer In Chief… COOTUS? You old coot you… )

What is is not in line with is the best interest of the typical middle class USA Citizen.

There we have the essential image of the incredibly out of touch Elite, preening and posturing and grinning like idiots, as they pilot their ship of non-state to destruction on the rocks of the angry Middle Class.

How does that go?
“If you are not part of the solution, you are part of the problem.”
I think that fits…

And the chemistry humor form:
“If you are not part of the solution, you are part of the precipitate.”
As the scum precipitates to the bottom of the vial…

Subscribe to feed

Posted in Political Current Events | Tagged , , , , | 33 Comments

Comparing and Choosing SBCs – computer on a board

“You can never cross the same river twice”

I don’t know who said it first. I know it has been said a lot since. For choosing computers, thanks to Moore’s Law and the compute power doubling every 18 months (original formulation) to 2 years (recent recasting – likely to be blown through by Vector Computing using Graphics Engines – GPUs as from Nvidia), it is very true.

The benchmark comparing the Raspberry Pi to the {whatever} from a couple of years back, must now contend not only with the Pi Model 2, 4 cores at 32 bits, but the Model 3 with 4 cores of 64 bits plus faster clock rate. Same price for all three… about a factor of 10 total uplift in performance available (though not always made available by the operating system… Early Model 3 ports run the Model 2 instruction set, so are 32 bit. Only now is effective use of the GPU for general computing starting to hit.)

So how in the heck can a person decide what to buy?

Well, you do it anew each time.

Look at your needs, find what fills them, compare prices and support communities, then pick one and jump in.

The good news (Very good news…) is that with price points about $45 to $55 ready to run, the ‘risk’ of a bad decision is minimal. Also note that in building out a ‘personal I.T. shop’ there are lots of places a ‘failed desktop’ buy can work wonders as infrastructure. Print spoolers / drivers. File servers. DNS Servers. Firewalls and Access Point Routers. Email servers. SQUID Proxy servers for web traffic, Intrusion Detection systems. It’s a very long list. I have most of those services running on an old, very out of date, single processor Pi Model 1 B+ and it is 90% idle much of the time. Cost? $35. Think of it as 7 coffees at Starbucks, or 1/2 tank of gasoline, or a “movie and snacks” for two. Or about 4 to 5 hours of time at minimum wage in the U.S.A. So don’t waste too much time optimizing the buy, as you are wasting time…

Besides, in a year your decision will be obsolete anyway.

My Bias

There are many design choices and personal preferences involved in picking an SBC (Single Board Computer). I can’t choose one for you, because what you want is different from what I want. So here are my biases. That will let you know where I’m making choices that you might not like.

1) “Friends don’t let friends drive Microsoft”… It is a PITA to configure. It is a security hole looking for a system cracker to make it his bitch. It is compromised from the factory to work with TLAs (Three Letter Agencies) in the USA under the PRISM program (and whatever it is called now). It’s a fat pig with painful attitudes (Blue Screen Of Death now a metaphor and “If it is having issues, have you tried rebooting?” as the #1 method of fix. So I have zero interest in the ability to drive the cut down midget Microsoft Windows for IdiOT device controllers.

2) I don’t run gadgets nor build my own cameras. (Yet…) So for those uses, the dinky Arduino actually works very well. I have a friend who builds robots and teaches robotics (retired from HP Engineer) and his students mostly use Arduinos for controllers. “Someday” I’m going to build a robot, but not now. Lots of I/O pins. Lots of add on daughter boards. Not very good at anything ‘desktop’ and graphics oriented… with low compute power (but overkill for most controller needs).

3) I like variety OS choices. I’m an OS Admin / Sysprog type at heart. I like to sample different ones. I like to have choices so I can use one for one tool and another for another use where it is better. Alpine for “network appliance construction” as it is intended for building routers. Debian / Devuan for more generic desktops. CENTOS or RHEL for ‘scientific computing’ as the tools are built in and well tested. Things like Plan 9 just to see what’s new and interesting. A poor-johnny-one-note SBC using something like Android is not going to float my boat. Now if you want to build your own Android Tablet, it could be ideal for you.

4) I’m a cheap SOB. Why pay an extra $10 for the same result? This, BTW, is largely why I’ve bought more Pi boards than anything else in the last half decade. They are not priced for a big profit, but for user available price points for kids and instruction. Now this comes with a performance price. Memory is a smidge too low. The I/O is limited (only USB 2 and shared with other I/O like ethernet that is only 10/100). It’s a weak machine for I/O intensive things like file servers or for memory intensive things like, well, most things. Yet it is “enough” for what I do. So why pay another $10 to $20 for an Orange Pi or Odroid that is better, but just means more hardware isn’t used to capacity on my desktop? If you do games, run big video edit operations, have a massive file server to support to a cluster of high performance compute boards, it’s just not the ticket. One SBC as a ‘good enough’ desktop for editing and browsing? It’s yours for $35, so why pay more?

5) I’m very suspicious of any hardware and software from China. No, I have nothing against Chinese. In grammar school and high school I had a crush on the Little Chinese Girl who’s family owned the Chinese restaurant in town. (We were born one day apart in the same hospital, went to school together for 12 grades, and both had restaurant families… and she was very cute ;-) But her older brothers did not approve of “round eyes”…) No, my complaint comes from the intensive State Sponsored effort by the Chinese Government to hack and compromise everything they can. Those nice electronic picture frames you plug into your USB to load pictures? At least one maker was found shipping them with viruses pre-installed. Another employer ordered something like 10,000 USB ‘thumb drives’ with custom plastics as a marketing promotional. Testing a few showed 1 in 10 shipped with a hacking virus included in the firmware… A team was assembled to sort out the infested ones and ship them back to the maker with a nastygram… I’d have cancelled the whole order and shipped them back, COD. But marketing didn’t want to change the event.. So, simply put: IF you buy electronics made in China, you are gambling. Period. Full stop. Some of the Pi boards are made in China, but the design and parts are UK and USA, and I presume that given the DIY community using them, ‘bad behaviours’ would be outed pretty quickly. For example, I’ve not seen any unexpected network traffic so far. Oh, and the firmware is a binary blob loaded at boot time by code in the CPU chip, all of which is American, so unlikely to be hacked. The other problem is China has lots of fraudulent substitution of substandard materials and counterfeit labels. When possible, avoid China for electronics. ( I know, ever harder to do. IFF there is intensive QA outside China, like by Apple, it can work out OK.)

6) My major interest is a simple functional desktop computer. Not games. Not audio / visual servers. Not robots. I do have a minor interest in I.T. Appliances.

Your List

You need to make a similar list of biases and desires. That is your roadmap to what you want. Don’t use mine, use yours.

Performance needed?

Applications to run?

Devices and appliances to build or operate?

Gizmos like cameras and tablets to build?

Frequent OS playing, or just want it to run one OS?

Do I need a large and helpful community with an active forum?

Am I happy to use a Beta operating system with minor “issues”? Not so minor?

Do I need a real clock? (The Pi has no hardware clock. It’s a bit of a pain at boot time…)


Once you do that, then the rest of this posting can point you at things to help you find what is right for you and for your application.

Some Resources

Realize that anything I say comparing two boards will be out of date a year after the board was first shipped, which often means immediately on posting. For that reason, every comparison and search needs to start anew from constantly updated resources. Wait 2 years and that “ideal dream board” is now “that nearly useless thing in the corner” (that I use for all sorts of fun infrastructure stuff ;-)

For all things Pi, start here:

As the original question came from PaulID (h/t) here:
and was asking about Pi vs Beaglebone Black, see:
for more on them.

For a general comparison of all things SBC, the Wiki does a pretty good job:

Do realize there are dozens of SBCs on that list, and even with constant updates, it is often a little out of date for the latest and greatest.

Comparison of CPUs is at best a black art. Then matching that to real world desired uses is even harder. But as a start, this wiki is helpful.

And just to make it more fun, each SBC maker often does minor revs of CPU chips or execution speed inside one board design and with only minor changes to the name / designation. A, B, B+ etc. etc. Just figuring out what you are buying once you have chosen a CPU and SBC board maker, design, and major model can be a challenge in watching single letter or special characters in names…

Oh, and all MIPS and all MHz are not equal. Millions of Instructions Per Second doing what? Integer? Float? Character manipulation? Graphics? MHz doesn’t map to Instructions Per Second the same way in all chip designs. One may ‘clock’ at 1 GHz and give 1 Gig instructions. Another 500 Meg. Another 2 Gig.

Then season with the fact that the memory to CPU balance matters (that Amdahl vs Pi link above) and that often I/O dominates real world use. More MHz doesn’t do much if you are limited on the memory or ethernet…

With that caveat, there are some fun benchmarks here:

Mostly from historical machines, but also recent PC hardware. I’ve not found ARM chips on it, though. Now it is really useful for dampening the urge to disparage some SBC as too slow to be useful… Especially when you figure out your Pi Model 3 is faster than a Cray from the 1980s… “Supercomputer for $35” anyone?…

For ARM and ARM vs Intel, there’s lots to choose from, and anything not updated in the present year is out of date… so a search is your best bet:

I find this one interesting:

This one is a bit dated, but has an ARM block:

For figuring out just which processor is a what, it will take time, but getting familiar with the ARM product line, byzantine as it is, is worth it:

I take a whack at demystifying it a bit here:

Comparing Pi and BeagleBone Black

I make no representation that I’ve got this right. ANY comparison will be full of individual bias and flat out guessing. Unless you compare the entire set of: Hardware, OS, application, compiler tool chain and libraries used for both, memory size and speed, disk size and speed, tendency to swap, etc etc on the exact same problem (that is not always possible and usualy not done) you can’t say for sure what actual performance will be.

This is just my eyeball spitwad while witling at it. OK?

First off, there are folks who have already done the comparison. They look to mostly be comparing the Black to the Pi Model 1, with a couple looking at the Pi Model 2. Well the Pi Model 3 is way better, and the same price as the old 2 was when I bought it… so “which Pi” matters a lot. The Pi 1 is a v6 instruction set, 32 bit, and essentially obsolete these days. The Pi 2 is a v7 and caught up with industry practice for a 32 bit. The Pi 3 is a v8 instruction set WITH a nice GPU that can do math too, if you compile things right. Oh, and both the 2 and 3 have 4 cores.

In the first Pi:

In the Pi Model 2:

In the BeagleBone Black:

In the Pi Model 3:

Now that move from ARM11 to A7 put the 4 core Pi-2 at a bit more performance than one higher end new design A8 core. Then the Pi Model 3 with the A57 left it in the dust. Especially for things desktop, which is what I do.

Oddly, this is made clear by the Beagle folks as they compare the Pi 2 to the Beaglebone. They do a great job of showing what they excel at (controlling things and being in the midst of interesting hardware projects) so I’m not going to rewrite it here:

I’ve bolded a bit I think matters to me:

Getting started with electronics projects quickly is pretty well optimized on the Bones and on BeagleBone Black we introduced an on-board eMMC flash (4GB as of rev C) that is pre-programmed with a Debian Linux distribution, out-of-box tutorial, IDE and everything you need to start your development. Part of our motivation to include this was the existence of forgery SD cards. The on-board eMMC has clear specifications on performance and reliability which are a must for doing any serious work. The higher bandwidth interface gives a nice performance boost. Having the flash pre-programmed saves significant time and money by delivering an immediate out-of-box developer experience without needing to visit the store again and download huge disk images.

The Bone also includes features like USB client and boot from USB/serial direct from the in-chip ROM code. The USB client is a key component to the unique rapid quick-start out-of-box experience shipped with every Bone. Plug it in and the exposed flash drive has all the documentation and drivers needed to work with the board from any computer. The board serves up a web site with a tutorial for learning to do physical I/O and an IDE for doing all of your development, even a command-line, right with your browser. Native tools and libraries ship for Python, JavaScript, C/C++, Ruby, Perl and others, but Java and an endless supply of others are only an ‘apt-get’ away. continues to be the innovator and the open partner with BeagleBoard-X15 already in the hands of beta developers, though it hasn’t yet been officially “announced”. That is a bit of the differences in the way we do things. The device documentation is already public and patches for BeagleBoard-X15 have already been pushed on the Linux mailing lists. When BeagleBoard-X15 does launch, it will have many times the performance and interface possibilities than Pi 2.

The Pi 2 is a pretty cool little affordable desktop computer with a business model that makes it attractive for those who want to use it as-is. For those who want to make cool stuff with electronics that includes an embedded Linux computer, they need to be sure to check out what the Beagles have to offer.

Last updated by on Wed Feb 11 2015 20:00:44 GMT-0000 (UTC).

The Pi Model 3 will just be more so…

This analysis is a bit old (2014) and isn’t the Model 2 or 3, but the general conclusions are right:


Now that we have looked at each category in detail, it is a simple matter to draw some conclusions about which circumstances should lead you to choose one board over the other.
When the BeagleBone Black is the Right Choice

Projects that need to interface with many external sensors – The incredible number of pins on the BeagleBone Black and the many bus options allow you to easily interface with pretty much any device out there.

Anything requiring small form factor but high speed processing – For example this super cool 33 node Raspberry Pi computing cluster would have been much better off using the BeagleBone Black, both from a price and performance standpoint.

Projects that you may wish to commercialize – Since the Raspberry Pi is more of a closed-source environment, it is impossible to make your own minimal versions. The open nature of the BeagleBone would allow you to just take the most important features and directly port that into your own design.

As an embedded system learning platform – The Raspberry Pi has its roots in education, but the fact that the BeagleBone Black works out of the box leads me to believe it is a better solution for learning about embedded systems.

For when you want it to “just work” – The fact that the BeagleBone Black works right out of the box is a huge bonus and allows you to get up and going in a few minutes rather than an hour or more.
When the Raspberry Pi is the Right Choice

Multimedia based projects – With the significantly more powerful graphics processing and larger number of connection options, the Raspberry Pi is a no-brainer for multimedia interfaces.

Community driven ideas – If you have a project that will in some way rely on the community for proper operation, you should choose the very active community of the Raspberry Pi. If you just think you will need support though, the BeagleBone community is very helpful and many Raspberry Pi projects will easily port to the BeagleBone Black.

As a graphical learning platform – Since the BeagleBone Black does not have quite the video capability of the Raspberry Pi, I would recommend the Raspberry Pi for learning about Linux in a graphical environment. Though to be fair you could do the same thing in a Virtual Machine, it just isn’t quite as much fun.
When Either One Works

Internet connected projects – If you want your project to send updates to a server, or maybe even act as a server, then either board should work just fine for you.

You just want to nerd out – Maybe you just want to get your nerd on. That’s okay, in fact it’s even becoming the cool thing to do. If that is your goal then either platform will serve you well.

I hope you found this guide helpful and that you will use it in making your next purchase. If you still can’t decide which one is right for you and you have some money to burn I really recommend just buying both of these systems. Each board has different strengths and they both offer something different. Happy hacking!

Now update that to add in quad core and 1.2 Ghz A57 and the Pi Model 3 pretty much dominates in anything desktop, cluster, or total computes needed. Price it at $35 and the $/perf is way high. But you want to make a ‘gizmo’, it’s likely the Beagle is your bet.

Now, some specifics:

Beagleboard Black

Processor: AM335x 1GHz ARM® Cortex-A8

4GB 8-bit eMMC on-board flash storage
3D graphics accelerator
NEON floating-point accelerator
2x PRU 32-bit microcontrollers

ONE core. A8 instructions, nice. 1 Ghz is OK. only 512 MB memory (but nice memory / core ratio). Onboard flash, so deduct $10 from the price for comparison to the Pi (or add $10 to the Pi for the SD card to make it go).

More than enough memory for a headless device. I blow through 1 GB with a few dozen web pages open in a browser window on the Pi. (Pages consume memory…)

Has NEON GPU / Float Vector unit.

Beagle adds 2 microcontrollers for offloading device management. GREAT for making gizmos and things like robots.


USB client for power & communications
USB host
2x 46 pin headers

LOTS of headers for device control and signaling. Ideal for devices and robots. A bit light on USBs so you will likely need a USB Hub for desktop use (but smaller and lighter for devices and embedded things). Has memory built in and boots ‘out of the box’ for folks who want to get to work building devices without fussing with the operating system. Runs a bunch if you are willing to update.

Pi Model 3:

Note the ‘replaced Pi 2’ in Feb. 2016? Might want to watch for a Feb. 2017 announcement… they seem to be on the one a year plan…

The Raspberry Pi 3 is the third generation Raspberry Pi. It replaced the Raspberry Pi 2 Model B in February 2016. Compared to the Raspberry Pi 2 it has:

A 1.2GHz 64-bit quad-core ARMv8 CPU
802.11n Wireless LAN
Bluetooth 4.1
Bluetooth Low Energy (BLE)

Nice that Bluetooth and WiFi are built in. 1.2 Ghz, so 20% faster per core. 4 cores… V8 instructionset. As operating systems are recompiled and debugged for the 64 bit arm64 instead of using the armhf 32 bit instructions, there will be an added performance boost. As of this moment, the one arm64 compile I tired (Devuan) “had issues”, so I’m typing this on an armhf Devuan Pi Model 3 and it is working fine.

Like the Pi 2, it also has:

4 USB ports
40 GPIO pins
Full HDMI port
Ethernet port
Combined 3.5mm audio jack and composite video
Camera interface (CSI)
Display interface (DSI)
Micro SD card slot (now push-pull rather than push-push)
VideoCore IV 3D graphics core

Double the memory (but 1/2 the memory per core, so a bit memory starved). LOTS of USB ports. Nice full sized HDMI so no hunting for a funny cable. Ethernet is 10/100 and shared with other I/O in the same chip, so a bit limited.

The Videocore gives it an advantage for things video. I’ve not done the research to see how good it is as a vector unit (math use of GPU).

IMHO, as a Desktop, the Pi Model 3 will blow the doors off the Beaglebone Black for lower cost. The Black will be a better device controller and those two microcontrollers will be golden in a robotic like device.

Sidebar on Others:

There are many other boards. I’ve looked at the Orange Pi, the Banana Pi, the Odroid family, and more.

IFF I really needed more performance, especially on I/O, the Odroids look to stake out that turf.

The Banana Pi looks to do the “clone of Pi but cheaper” and is made in China. Once at $35, I don’t see the need to be cheaper…

I’m not sure what niche Orange Pi is going for.

I really like the Cubieboards, but can’t find a use case that justifies paying up for them, for me.

Then there are the $15 and under mini-boards. Not good for a desktop, but at that price, making a small cluster to play with becomes very attractive… I’m resisting the urge until I can show I’m using all the computes I already own ;-) But a 16 board Beowulf for $160 to $240 sure sounds like fun ;-)

How Low Priced an ARM SBC?

At the other extreme, there’s this one:

Yeah, you read that right. $7 for a computer…

IoT-oriented $7 Orange Pi Zero has both WiFi and Ethernet
Nov 3, 2016 — by Eric Brown —

The 48 x 46mm Orange Pi Zero runs Linux or Android on a quad-core Allwinner H2, and offers WiFi, 10/100, microSD, USB host and OTG, and a 26-pin RPi header.

Shenzhen Xunlong has added a Raspberry Pi Zero competitor to its Orange Pi lineup of open source hacker boards. The new Orange Pi Zero is selling for just $7 plus shipping, for a total price of $10.30 when shipped to the U.S.
Whereas the Orange Pi Lite and Orange Pi One use the Allwinner H3 SoC, the Orange Pi Zero offers the rarely seen Allwinner H2 variant. As far as we can see, the only notable difference between the SoCs is that the H3 supports 4K video while the H2 is limited to 1080p. Like the H3, the H2 has four Cortex-A7 cores clocked to 1.2GHz, as well as a Mali-400 MP2 GPU clocked at 600MHz. The H2 is still much more powerful than the 1GHz, single-core ARM11 Broadcom BCM2835 on the Raspberry Pi Zero.

But, for me, for now, I’m happy with the Price / Performance of the Pi family (even if performance isn’t as good as some, and Price / Perf less good than others). I’m very familiar with it. I like being able to move the mini-SD chip to change the OS, so on-board memory for the OS isn’t attractive to me. I’m also more comfortable with US designed chips and UK designed boards with minimal micocode that only loads the boot loader so the actual software is US sourced. No Chinese Binary Blobs need apply… (Yes, full blob-free would be better…)

So that’s what I buy. Doesn’t mean other choices are wrong. Only means I’m right for me. YMMV with your use case and wallet.

Subscribe to feed

Posted in Tech Bits | Tagged , , , , | 3 Comments

Well that was fun – Mac Hell…

Where have I been today? After writing the article on Hacks & Cracks & Leaks (Oh My!), I entered Macintosh Hell.

The spouse is on a Macbook Air. It is about 5 years old. Last week it started acting strange. Processes would sometimes hang. What was on her desktop was visible in a terminal window (an ‘ls’ would show it) but didn’t show on the desktop. Sometimes the boot would fail. We made an appointment with the “Genius Bar”…

A Few Days Later

Yesterday we were AT the “Genius Bar”. After waiting about an hour after we arrived at our appointment time, we were seen. The guy doing the service was great. It all went well. It took about an hour? At the end of it, we had a working Mac again, and the price was “free”. Seems they don’t charge for ‘fixes’ at the “Genius Bar”, only for parts. OK… That’s the good.

The Bad was that he figured it was the disk as their equivalent of an fsck (File System ChecK) application found a lot of disk errors. Asked if we had a backup. I stated I’d made a couple that had some odd behaviours, but it was likely that one of them was good, and we had a roughly 3 month old one known to be good. ( I’d made a ‘tar’ copy that complained about errors but seemed to work, and a ‘cp -r’ copy that got into what looked like an infinite decent and was not converging to completion, so I killed after seeing the same error on the same file name about 5 times…) So, without further ado, he nuked (formatted) the disk and reinstalled the OS.

Well, I’d have liked to ask if he had any special way to make a backup, but that option wasn’t presented. I’d have liked to ask if there was an intensive disk check that could be done before the OS install, but that opportunity wasn’t presented prior to the OS install already being underway. Oh Well.

I had a working OS and a working Macintosh and all it cost was time. OK.

I also got to observe what the Mac specific tools looked like and what the behaviours were. That came in handy today.

Today I was to restore from our disparate backups what files we had.

I’d picked today to stop coffee, and was fairly useless until about noon when I gave up and had 2 cups.

I sure picked a bad day to give up coffee…

I started doing the restore. After copy of some modest bit via “cp” I realized I’d not done “cp -r” (in a terminal window) and stopped it with “Ctrl-C”. (The Mac, being basically a Unix under the skin, takes many / most Unix commands). At that moment, with control returning from the cp command to the OS, it crashed.

On attempted boot, it just didn’t see any disk at all. No complaints, just a white screen with a folder icon in the middle with a blinking question mark.

OK, now what? Back to the “Genius Bar” for another 2 hour wait to get another appointment with a 3 day wait to get service an hour after showing up? Ah, no. So I went digging and learned some new tricks.

By about 1 PM I had it loading diagnostics from the internet. The hardware check passed with flying colors. The LONG hardware check that takes near an hour passed without error. Disk Check could not see the disk. Zippo. Nada. Attempting to force a disk check at boot didn’t work.

My guess is the disk is flat out fried. Not much you can do at that point without a disk. I also suspect that an intensive disk check before installing the new OS at the “Genius Bar” would have caught it…

By about 2 PM I had a USB disk cleared, and used Disk Check to format it for HFS (Macs file system). Shortly after I had the default operating system being installed over the internet. As it is large, it took a couple of hours.

A bit before dinner time I had it booting off the external disk. The Usual account set up, agree to {whatever} or your machine doesn’t run, click this, select that yada yada… Then it is live. YAY!!! A Live Mac!!

It worked fine off the external disk. Reboots work too ;-)

Then came the restore of files, again and at last.

I un-tarred the tar file and got what was about what was expected. 9 GB of “stuff”. All the expected directories looked to be there. I copied back the “cp -r” copy and the USB 3 month old set. The old set looked as expected. But the “cp -r” copy had a 50 GB or so “Library” folder where the tar Library folder was only about 6 GB. This doesn’t show up in the “Finder” (window manager / GUI file display thingus) at all, so is ‘special’. It looks to be filled with cache and similar bits for various applications. The folders (directories) that came after it in alfa order were NOT in that backup (no surprise, really, as I’d killed it when looping on Library).

In the end, it looks like I’ve got all the user files, in one spot or another, and they were copied back to the external disk in the user home directory. Now I’ve got a Mac that seems to be working fine, with most and hopefully all the user files back in place. (Tomorrow I get to start on assuring the Applications are all there… something like writer and numbers or some such).

With that, it’s about 9 PM. Essentially 8 or 9 hours in Mac Hell. My neck is stiff, my back a bit sore between the shoulder blades. Muscles a bit stiff.

I sure picked a bad day to give up Aspirin…

But I’ve recovered things to a usable degree, mostly. Which is good because the spouse made another appointment with the “Genius Bar” for Saturday (earliest available) for them to decide the disk is dead and install a new one. Not only does having the Mac working from USB disk mean the Spouse can give me back my tablet and use her Mac (until Saturday) but it also means I can show them the great diagnostic of the Mac working FINE off of an external USB disk and not seeing the internal disk at all. Ought to make diagnosis easy for a Genius at the Bar…


Well, the good news is it looks like few to no documents were lost. The Spouse did a cursory look over and saw no problems. The detail compare, folder by folder, between the three copies will wait until later. A quick “du -ks” on the copies I made showed the docs and desktop folder equal sized, so “I have what I have” from the new set, and anything missing will likely be on the 3 month old copy set.

Also, only generic Apple Applications were being used, so that’s easy to recover (if I can’t find them on the 3 month old backup).

And the Very Good News is that now I can put “Current on Mac OS X – rebuilt disks, reinstalled OS, recovered files” on my resume… It’s been a good decade since I had to fix a Mac. Things have changed. But not too much ;-)

The very bad news is that I’m out of both Scotch AND Wine… and the only beer in the house is 2 six packs of “Chum – the San Jose Sharks hockey team ‘red ale'”… bought as a gift for my son, so off limits.

I sure picked a bad day to run out of booze…

Subscribe to feed

Posted in Human Interest, Tech Bits | Tagged , , , , | 22 Comments