Devuan and Climate Cluster Milestone

Just a short note on some hardware status.

As of this moment, I’ve got 2 x Pi Model 2 boards, in the dogbone stack / case, running on a mostly built as I like it Devuan Jessie. My desktop at the moment is a similarly built Devuan, but with the hardware being my Pi Model 3. That’s a total of 12 cores available for use in a cluster formation.

They all have distcc installed (though not yet configured) so I can now do compiles of code spread over many cores. But I’ve set it up before, so configuration won’t take long.

All of them were made from a NOOBS base installation of Raspbian, then upgraded to Devuan.

I then ran the script that I’m including below. Note that many bits are commented out. Some of them I just didn’t want on these systems (like a VNC remote terminal). You can customise this list to suit your needs. For example, I’ve got gfortran and distcc stuff in it. If you are not going to be doing big compiles or FORTRAN on your system, you don’t need them.

Here’s the “disk” (chip) usage:

Filesystem     1K-blocks     Used Available Use% Mounted on
/dev/root       15951288 10406200   4711760  69% /
devtmpfs          469532        0    469532   0% /dev
tmpfs              94776      416     94360   1% /run
tmpfs               5120        4      5116   1% /run/lock
tmpfs             189540    47624    141916  26% /run/shm
/dev/mmcblk0p6     67434    20698     46736  31% /boot
/dev/mmcblk0p9  13157084    34988  12430692   1% /Climate
/dev/mmcblk0p8     67434        0     67434   0% /boot0
/dev/mmcblk0p5     30701     1563     26845   6% /SETTINGS
cgroup                12        0        12   0% /sys/fs/cgroup
cgmfs                100        0       100   0% /run/cgmanager/fs
tmpfs              94776        0     94776   0% /run/user/1000

Note that I have this on a 32 GB “chip” (mini-SD card) and have it set up with a /Climate directory where I’m presently working on GISS GSM Model II. Don’t let the size of “/” worry you. I had to dump a 6 GB USB thumb drive into that for a different use… The actual install takes about 6 GB and it fits on an 8GB ‘chip’ if you don’t make a /Climate directory. (One of the Headless nodes is using an 8, the other a 16, this workstation node has a 32 since this is where source and compiled bits will live.)

There is still a fair amount of configuration to do since I made these from the ground up. Things like turning on distcc, enabling ssh so I can remote log in (just found that out when I brought them up headless…) making sure they are booting with fixed IP (right now, one is, the other isn’t) and have a set name (“Headless1” and “Headless2” are now set). That kind of stuff.

One big bit I need to do: These by default boot up live with a GUI open as user Pi. I want to have a login prompt that asks for a password AND my “usual and customary” login name and home directory… So that is yet to do.

I tried to do a Linpack performance benchmark, but it would reach 396% of CPU and hang. That was on a Debian SystemD chip, so I hope it doesn’t hang on these. (It was after that I ‘moved on’ to doing the cluster build).

The boards are just plugged into the ethernet ports on my interior WiFi router for now. As the Pi is only 10 / 100 Mb Ethernet, that Netgear router is plenty fast for them. When I go over 4 (IF I go over 4) I’ll need to fire up my 8 port Netgear Switch…

Moving Forward

I’m now in a position to try running the Model II GCM code, and then I can start looking at Model E (that has distributed MPI already coded into it). If needed, I can put similar changes into Model II and make it distributed execution (though I’d rather not just yet).

Most importantly, I can make some needed tests of MPI function and efficiency. OpenMP (thread parallel) was dismal in an earlier test on Debian, it actually ran slower! Hopefully MPI (MPICH2 – core parallel) will actually have speed increases…

Essentially, I have my Toy Cluster built and ready to play with, test, and characterize. I think it will be doing “interesting things” by Sunday. (Spouse has spoken for Saturday as Macintosh Fixing Day… new disk, load data, etc…)

So far there is only one “bug” I’ve seen in this release of Devuan. When you choose to logout, it gives a panel with choices of “shutdown” and “exit to prompt” or some such. Shutdown doesn’t work, so you must chose the exit to terminal line prompt. That then doesn’t give you a terminal line, but another panel where shutdown does work… Petty annoyance, so might take a while for Devuan to fix on the Pi.

Overall, I’m quite happy with it, and I’m very happy to be done with the long search for a way out of systemD hell.

The Build Script

Here’s my “build script” for the Devuan installation. As is my custom, I make one of these for any build I expect to do more than once. It references earlier postings on the earlier steps (doing a Debian installation) and points at the reference for the Debuan upgrade that I found most useful (when making my first Devuan chip in the article here )

The Script:

root@raspberrypi:/media/pi/PINK_ext4/Pi_Config_Files# cat BuildDevuan 
echo " "
echo "Do the NOOBS install: "
echo " "
echo " "
echo " "
echo "Do the Raspbian installation.  Then copy this script from an external SD"
echo "card or USB drive into your working directory (home directory or /media/data"
echo "after all the usual and customary NOOBS setup questions are answered and done."
echo " "
echo "Make sure you select the USA keyboard if in the USA, default is UK!"
# In general, I'm encapsulating what all I did in these two postings as a script:
# If you didn't already change the password while running NOOBS,
# When done, log in as 'pi' password 'raspberry'.  Change the password.
# passwd
# and respond with the new one when prompted.

echo ""
echo "Do the upgrade / change to Devuan from this posting:"
echo ""
echo ""
echo " "
echo "It basically consists of changing /etc/apt/source.list to have:" 
echo ""
echo "deb jessie main"
echo ""
echo "as the only active line. Then do:"
echo "apt-get update"
echo "ignore the error message about keyrings and signatures, you don't have it yet."
echo "apt-get install devuan-keyring"
echo "Now you have the keyring... and can do another apt-get update if you want one"
echo "with no error messages, but it doesn't matter."
echo ""
echo "apt-get install sysvinit-core"
echo "apt-get install base-files"
echo "That last one may say you are already up to date."
echo "The next one will kill Berry Boot images with an attempted kernel swap."
echo "apt-get dist-upgrade"
echo "apt-get autoremove --purge"
echo "# apt-get autoclean"
echo ""
echo "Now reboot to start running Devuan properly.  If you don't have xfce, add it"
echo ""
echo "apt-get install xfce4"
echo "and startxfce runs it." 
echo ""
echo "Also, to change the name of your machine, edit /etc/hostname and make it"
echo "what you like.  "
echo "Here, I'm going to just set mine by brute force write to the file."
echo " "
echo "echo 'Devuan_armhf' > /etc/hostname "
echo " "

echo 'Devuan' > /etc/hostname 

echo " "
echo "Next, do the 'usual' update upgrade that brings you up to the present"
echo "repository status (need a network connection from here on out)"
echo " "
echo "You can either put 'sudo' in front of each of these commands, or just "
echo "'become root' which is what I usually do."
echo " "
echo "sudo bash"
echo " "
echo "then run this script with ./BuildIt (assuming you didn't change the name"
echo "and that you are 'in' the directory where it is located.)"
echo " "
echo "apt-get update"
echo "apt-get upgrade"
echo " "

#apt-get update
#apt-get upgrade

echo " "
echo "Start doing useful operational 'packages'. "
echo " "

# This gets the useful tools like "nslookup" for looking at Domain Names

echo " "
echo apt-get install dnsutils
echo " "

#apt-get install dnsutils

echo " "
echo " VNC is a nice way to get a remote desktop.  It takes some configuring later."
echo " "

echo " " 
echo apt-get install tightvncserver
echo " "

#apt-get install tightvncserver

echo " "
echo "I like wicd for an easier way to manage wireless devices and networks."
echo " "

echo " " 
echo apt-get install wicd
echo " "

#apt-get install wicd

echo " "
echo "Scrot is a tool for taking screen shots by saying 'scrot' in a terminal"
echo " "

echo " " 
echo apt-get install scrot
echo " "

#apt-get install scrot

echo "Normally I would install 'build-essential' to get things like C compiler"
echo "and some language tools, but they were already installed on the R.PiM2."
echo "Now I add gfortran and distcc"

#apt-get install build-essential

apt-get install gfortran distcc distcc-pump dmucs ccache doc-base gfortran-4.9-doc libgfortran3-dbg

echo " "
echo "Some 'user land' useful things like browser options and Office / Mail tools."
echo " "
echo "Chromium is the 'chrome' browser from Google but in Linux land"
echo " "
echo "    CHROMIUM is installed by default now"

echo " " 
echo apt-get install chromium
echo " "

#apt-get install chromium

# IceApe is a "more free" version of IceWeasel that is a "more free" version of
# Firefox that is a rebranded Mozilla that is...   IceDove is the matching
# Thunderbird replacement minus the trademarks, non-free bits, etc.

echo " "
echo "Doing IceApe browser and IceDove mail reader"
echo " "

echo " " 
echo apt-get install iceweasel
echo apt-get install icedove
echo " "

apt-get install iceweasel
apt-get install icedove

echo " "
echo "GIMP is the photo editor ( 'photoshop Free'...) "
echo " "

echo " " 
echo apt-get install gimp
echo " "

apt-get install gimp

echo " "
echo "Don't forget Libreoffice - Microsoft?  We don't need no steenking MicroSoft..." 
echo " "

echo " " 
#echo apt-get install libreoffice
echo "Already installed by default"
echo " "

#apt-get install libreoffice

# I tried "arora" and got error messages and "xbmc" was not working for sound
# so I'm not installing those again until it's clear they work.  Arora is a
# browser (so who needs "yet another browser" with Epiphany in by default and
# with both IceApe and Chromium installed?...) and I'll likely make a dedicated
# SD card for the media center option as there are 2 Pi Model 2 version up
# and I won't need to screw around with Debian issues...
#apt-get install arora
#apt-get install xbmc

# As I also wanted one of these to be a bittorrent server, I sometimes add
# the "transmission" bittorent code.

echo " "
echo "Adding the 'transmission' bit torrent server"
echo " "

echo " " 
echo apt-get install transmission
echo " "

apt-get install transmission

echo " "
echo "To get NTFS disks (like USB or an NTSB formatted SD card in adapter) to "
echo "work 'read write' instead of just 'read only', you need ntfs-3g"
echo " "

echo " " 
echo apt-get install ntfs-3g
echo " "

apt-get install ntfs-3g

# In Theory, this installed 2 VNC "viewers" so the R.Pi could use VNC to 
# get to other machines.  In practice, I found that one of them locked up
# my console when launched against my own machine as target (might be a 
# PIBKAC problem - Problem Is Between Keyboard And Chair - as the R.Pi
# isn't really expecting to drive 2 video sessions at once (the real one
# and the VNC one inside the real one...) so maybe all is fine and I just
# need to RTFM (Read The, er, "Friendly" Manual) before using software...

echo " "
echo "Some VNC Viewers for being the client instead of the server"
echo "I've not used either of these yet so have no clue about them in practice"
echo " "

echo " " 
echo apt-get install xtightvncviewer
echo apt-get install ssvnc
echo " "

#apt-get install xtightvncviewer
#apt-get install ssvnc

echo " "
echo "Want an NFS (Network File System) server so you can share disks with" 
echo "your internal network?  This will install the code, then you get to" 
echo "configure things like /etc/exports"
echo " "

echo " " 
echo apt-get install nfs-kernel-server
echo " "

apt-get install nfs-kernel-server

# prior to first use.  Or reboot.

# In your /etc/exports file, put something like:

# /etc/exports: the access control list for filesystems which may be exported
#		to NFS clients.  See exports(5).
# Example for NFSv2 and NFSv3:
# /srv/homes       hostname1(rw,sync,no_subtree_check) hostname2(ro,sync,no_subtree_check)
# Example for NFSv4:
# /srv/nfs4        gss/krb5i(rw,sync,fsid=0,crossmnt,no_subtree_check)
# /srv/nfs4/homes  gss/krb5i(rw,sync,no_subtree_check)

# /YourFileSystem  *(rw,sync,fsid=0,no_root_squash)
# But without the # in front of YourFileSystem... and with your file system...

echo " "
echo "IF you chose that 'add a 512 MB partition option' at build time"
echo "This adds it to the /etc/exports file so it is NFS mountable elsewhere"
echo " "
echo "echo '/media/data   *(rw,sync,fsid=0,no_root_squash,no_subtree_check)' >> /etc/exports"
echo " "

#echo "/media/data   *(rw,sync,fsid=0,no_root_squash,no_subtree_check)" >> /etc/exports

# Remember to do a 

echo " "
echo "Restarting the appropriate services so NFS will work"
echo " "
echo " " 
echo service rpcbind restart
echo service nfs-kernel-server restart
echo " "

service rpcbind restart
service nfs-kernel-server restart

# I also made my box a static IP number as it's a server.  You will need to
# make this your own server name and IP numbers.
# Here's my /etc/network/interfaces file with leading # to make it comments.
# I will make this a "dump these lines in to replace" in my running version.

echo " "
echo "Remember to make your /etc/network/interfaces file have a static IP#"
echo "If you are going to be using PXE boot and such"
echo " "

#auto lo
#iface lo inet loopback

#auto eth0
#allow-hotplug eth0
#iface eth0 inet static
#dns-domain chiefio.home
#auto wlan0
#allow-hotplug wlan0
#iface wlan0 inet manual
#wpa-conf /etc/wpa_supplicant/wpa_supplicant.conf
#auto wlan1
#allow-hotplug wlan1
#iface wlan1 inet manual
#wpa-conf /etc/wpa_supplicant/wpa_supplicant.conf

# Don't forget to do a
# ifdown eth0
# wait a minute for it to quiet down
# ifup eth0

# As I want this to be a DNS server, DHCP server, and PXE server (uses a 
# tftp or "Trivial File Transfer Protocol" server, all of those can come in
# one package with dnsmasq.

echo " "
echo "Installing a light weight but effective DNS, DHCP and TFTP service"
echo " "

echo " " 
echo apt-get install dnsmasq 
echo " "

apt-get install dnsmasq 

echo " "
echo "Yes, it takes configuring.  See the file at"
echo " /etc/dnsmasq.conf"
echo " "

# Then I installed the Apache web server :


echo " " 
echo "Instlling the Apache Web Servier and related stuff"
echo " "

echo " " 
echo apt-get install apache2 apache2-utils apache2-doc
echo " "

apt-get install apache2 apache2-utils apache2-doc

# and yes, it takes some configuring and even web page building.
# See files in places like /etc/apache2/sites-available and more.

echo " " 
echo apt-get install libapache2-mod-php5 php5 php-pear php5-xcache
echo " "

apt-get install libapache2-mod-php5 php5 php-pear php5-xcache

echo " " 
echo apt-get install php5-mysql
echo " "

apt-get install php5-mysql

echo " " 
echo apt-get install mysql-server mysql-client
echo " "

apt-get install mysql-server mysql-client

# These needed to be added for Transmission and for file systems

apt-get install transmission-daemon
apt-get install btrfs-tools xfsprogs gparted f2fs-tools unionfs-fuse
apt-get install hfsplus hfsutils
apt-get install squashfs-tools aufs-tools

echo " "
echo "And that's the end of my present install build process."
echo " "
# There are several files to edit and configure.  Eventually I'll add a 
# "here script" to dump them from this script to where they belong, or 
# I'll just save a copy and have a 'save / restore' copy process.
# Once I get everything configured ;-)

Folks may notice this looks mostly the same as the Debian build script. Some things commented out. Some added. A few, like iceape, removed since there is not target for it to build anymore.

For anyone looking to get a working Devuan on a Pi, with most of the goodies needed to do interesting things, this ought to get you there.

FWIW, it took me about 5 hours from start to end. (Part of the duration will depend on network speed… especially that part where the apt-get dist-upgrade says to go get a cup of coffee…) I made copies of the chip at the NOOBS installed stage, at the Devuan raw upgrade, and after my scripted added packages. (dd bs=10M if=/dev/sd{your drive letter} of=/where/you/save/it) using the 8 GB chip for the basic build, so less to store. Then later did a similar dd from the final saved image onto the 16 GB chip. That leaves about 8 GB not allocated, so with gparted you can make that /Climate partition…

During much of that time you can be doing other things. Do realize that during the Devuan upgrade it does prompt a couple of times, and at one point tosses you into a log file reader. I think I did :q to get out of it by vi instinct, and who knows what the proper exit command is… so it isn’t entirely ‘fire and forget’, but more like “fire and check back”.

Now, with my saved chip images, I can skip all that in the future and dd it onto any new chips I want to make, then modify the config.

With that, it’s time for a break! Happy Hacking!

Subscribe to feed


About E.M.Smith

A technical managerial sort interested in things from Stonehenge to computer science. My present "hot buttons' are the mythology of Climate Change and ancient metrology; but things change...
This entry was posted in AGW and GIStemp Issues, GCM, Tech Bits and tagged , , , , , . Bookmark the permalink.

12 Responses to Devuan and Climate Cluster Milestone

  1. E.M.Smith says:

    Just an FYI on the Chrome Browser:

    It chews memory like crazy. I had, maybe, 6 tabs open and only 2 of them were things like heavy page weight news sites, the rest being my usual blog maintenance pages. The Pi Began To Swap… the full 1 GB was used up AND there was about 90 k on swap.

    Now I’d not yet put my hard disk swap area on the Pi, so it was swapping to the SD card.

    PAINFULLY SLOW. At first I thought the system had hung, even clicking to another window didn’t move, at first. Then I noticed that my “top” panel (I always run ‘top’ in a terminal window and scan it for ‘unexpected’ logins or processes) was showing updates to process running and memory used. Also, then, that the swap daemon was chewing about 9% of CPU.

    Now this on a system that used about 300 MB without any browser open, but a few terminal windows. So it was all the browser causing the memory suckage and the swappage. (On fresh boot, it has 200 MB of memory used before any browser launch, but with an X session).

    Now, same system, running IceWeasel (now a link to FireFox-esr … wonder what esr stands for… but nevermind…) with 6 tabs open (all blog related – one the giant Tips page) I’ve got 750 MB memory used, nothing on swap.

    Lesson learned? On any system where you do not have a real swap on disk space set up, using the Chrome Browser is going to be an exercise is small open page counts… or long pauses. (But at least now the Chrome browser does work… in earlier Raspbian releases it was there, but crashed; then it was removed…)

    This also implies that the compute nodes need to stay under max mem or they will bog down too. I don’t see that as a problem (yet) since they will not have the big memory suck of X-Windows or browsers at all. But I’ll need to watch for it in early runs. I don’t want to put a $25 powered USB Hub and a $50 disk on each $25 board just to avoid swap issues…

    FWIW, very sloooooo-lyyyy the browser responded to my click on the TIPS page and the two news articles with the highest page weight and let me close them. As that was done, memory use dropped dramatically. I suspect Chrome tries to cache in memory everything on the pages for fast snappy response; but that fails when memory limits. Other browsers seem to cache more to disk and treat memory as more limited.

    A good example of characterizing resource use in a code, and choosing hardware to match it (or changing to a different code). IF I were using the Chrome browser regularly, I’d need a machine with at least 2 GB of memory, and preferably about 4 GB (given the number of tabs I like to have open) and the Pi would not be it. It is “memory limited” for this program (but not for FireFox and other browsers…)

    My Desktop Workstation config includes a powered Hub and real USB swap disk. While not done yet, my intent is to put my home directory on a real disk partition too, so things I do in it (like compile large programs) can avoid SD-chip related write time lags. (Reads are often very fast – even 80 MB/sec, but writes are very slow as to write one byte, it writes an entire large segment of the chip, so must do a ‘read, modify byte, write’ of that slice). I’ll also have an NFS mounted partition on the worker nodes so their writes to disk go over ethernet to a Real Disk ™ and are not chip limited. Swap over NFS is a risky and discouraged thing, but I might explore it anyway… can’t be much worse in performance than swap to SD…

    So there you have it. First performance tuning discovery of the system. Pay attention to memory use and avoid swap at all costs, unless you have put it on Real Disk. Also note that “write to disk” intensive applications may limit on SD write speed and benefit from network mounted file systems on Real Disk. I/O limits hard when it runs into SD card write speeds. (It would probably be of benefit to benchmark write speed for the variety of chips I have, but my plate is overflowing at the moment, so that will come later and only if really needed – doing a web search for SD write benchmarks is a “Dig Here!” for later.)

    I’m going to change swappiness ( /etc/sysctl.conf add line vm.swappiness = 20) and increase the memory threshold for when the OS starts prepping to swap (so it is less likely to run out of memory while trying to swap) by also adding “vm.min_free_kbytes = 32768” to the same file. That ought to get low usage bits of pages swapped out earlier (or at least started out earlier), given that we now know swapping will happen with the default use of Chrome.

    Oh, and I’m not sure this is why, but I found this tidbit on the Model 3 that might explain some of the buggy bits in the arm64 test I did:

    Enable ARMv8 on RPi3B

    When running lscpu or cat /proc/cpuinfo the reported processor ist ARMv7. To tell the RPi3B to start in ARMv8-mode you have to add a new line to config.txt.[10]


    Or it might only have effect on the lscpu and cpuinfo stuff… but interesting to know.

    For now, I’m staying with v7 mode and armhf build.

  2. E.M.Smith says:

    Oh, and I shut off the automatic login as Pi. In the file:


    put a # in front of the autologin-user = pi like this:


    then reboot and you get a login prompt instead of anyone who boots being in.

    This uses the lightdm display manager, and I’m pretty sure that by shutting it off (somewhere…) I can get back to the “just give me a prompt, no Gui, no Wallpaper, no nuttin’… ” That’s what you want on headless nodes as they all take a bit of memory and computes…

    says to just do:

    sudo update-rc.d lightdm disable

    and since that is using the rc.d form (systemV init) and not the systemD stuff, I think it is right. We’ll know after I try it and reboot ;-)

  3. E.M.Smith says:

    Well, this is an odd one…

    X worked fine for user Pi. It worked not at all for my user-id (do an ‘adduser’ and answer questions to make your personal account. I added mine by hand so I could set the UID and GID to what my userid number was on my prior systems… to avoid needing to chown it all and stay compatible with the old chips).

    Digging in the error log, it couldn’t find a display and complained about /dev/fb0

    I had to do a “chmod 666” on it to make it world read /write and then “startx” would work.

    UPDATE: Looks like /dev/fb0 is “group video”, so the proper way to make this work is to add any new user to the /etc/group file video group:


    End Update.

    Now the fun stuff… for one new user-id, the display looks like the one for Pi. Menu at top and all. For my userid, once I mount my home dir from the USB, it looks like it always did under LXDE with menu at the bottom and menu not showing Firefox but showing IceWeasel… OK…

    So there’s some more X details to iron out between the Devuan defaults and the config in my home directory… Don’t know if it is that Devuan is using Openbox or what, but it works both ways in any case.

    BTW, this kind of “permissions problem” and “it works as shipped” is common in young ports. Still, I’m going to boot one of my older Debians and see what permissions it has set on /etc/fb0 …

    In any case, I’m now “good to go” with my stuff on Devuan and looking just like it did on Debian…

  4. E.M.Smith says:


    So a fresh install has it nagging me at each Pi login that the default passwd on Pi wasn’t changed and that makes ssh login a HUGE risk and I need to do something NOW!! about it.

    OK, I changed the Pi password.

    Then, after all set up headless I find I can’t ssh into the system… That’s different, I think.

    Searching, I find this:

    As of the November 2016 release, Raspbian has the SSH server disabled by default. You will have to enable it manually.

    To enable SSH on machines through console:

    Enter sudo raspi-config in the terminal, first select advanced options, then navigate to ssh, press Enter and select Enable or disable ssh server.

    To enable SSH for headless machines:

    For headless setup, SSH can be enabled by placing a file named ssh, without any extension, onto the boot partition of the SD card.

    Oh Joy, a full on raspi-config run just to make ssh go? No thanks… and just why did they break it in the first place and THEN nag me that it was a risk? Sigh….

    So now I get to either shutdown my workstation and put the head bits on the headless servers and config them, or shut them down (via the only available option of a powerfail…) and put a Magic File on the SD card… So I’ll search for more specific magic sauce to turn on ssh without a full on run of raspi-config…

    Well see if this has persistence:

    root@Devuan:/# sudo service ssh start
    [ ok ] Starting OpenBSD Secure Shell server: sshd.
  5. Paul Hanlon says:

    Thanks for that, ChiefIO, I’m definitely going to try Devuan (and Alpine, as soon as I can find a use for it). I’m surprised too that they disabled SSH in that way. I find now that I have to use one of those FTDI USB cables connected to the GPIOs to configure a Pi now, which isn’t a game-changer, but is inconvenient when going headless.

    I have a Toshiba laptop for doing that, which doesn’t restrict the USB output to 500mw, so I can power the Pi through the GPIOs as well, and it works fine, at least for setting it up headless. Also I set up the network entirely through /etc/network/interfaces, which bypasses wpa_supplicant.conf altogether. Seems to work just fine, especially when I use the directive wireless-power off (could be wireless-power=off). Counterintuitively, this means that the Pi doesn’t need a reboot when it cannot connect to the network.

  6. E.M.Smith says:

    @Paul Hanlon:

    You’re most welcome. Most of the work was done by others, I’m just showing what it took to get it the way I wanted it… but “many hands make light work”.

    Per SSH, see below… You can make it “go again”:

    And another one… Looks like Debian really decided to lock up ssh security after the IdiOT device based DDoS attack, so now by default it won’t work if you have just the password and definitly not if you change a hardware identity via swap of chip, which I do a lot.

    Still not sure if the sshd start will survive a reboot, but while it is up, tried to ssh in. What always worked, failed…

    ssh -l gcm
    Someone could be eavesdropping on you right now (man-in-the-middle attack)!
    It is also possible that a host key has just been changed.
    The fingerprint for the ECDSA key sent by the remote host is
    Please contact your system administrator.
    Add correct host key in /home/chiefio/.ssh/known_hosts to get rid of this message.
    Offending ECDSA key in /home/chiefio/.ssh/known_hosts:8
      remove with: ssh-keygen -f "/Pink/ext/home/chiefio/.ssh/known_hosts" -R
    Password authentication is disabled to avoid man-in-the-middle attacks.
    Keyboard-interactive authentication is disabled to avoid man-in-the-middle attacks.
    Permission denied (publickey,password).

    So I’m forced into constant “key maintenance”… or I have to undo the done…

    in my home directory, .ssh/config, I’ve put:

    ServerAliveInterval 600
    StrictHostKeyChecking no
    UserKnownHostsFile /dev/null

    The first line I’d added way back. The next one lets me use just a password to log in, skipping the key generation stuff, and the third one avoids conflicts with the saved keys from prior logins.

    All so I can get a terminal window from my desktop to my Pi Stack 2 feet away, behind two levels of routers / firewall and with IDS running in the middle… Sigh. Sometimes there’s too much security for functionality to function…

    But at least now I can ssh in…

    ssh -l gcm
    Warning: Permanently added '' (ECDSA) to the list of known hosts.
    gcm@'s password: 
    Linux Headless1 4.4.38-v7+ #938 SMP Thu Dec 15 15:22:21 GMT 2016 armv7l
    The programs included with the Debian GNU/Linux system are free software;
    the exact distribution terms for each program are described in the
    individual files in /usr/share/doc/*/copyright.
    Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
    permitted by applicable law.
    What are you doing here?  You don't belong here, get out!
    Last login: Sat Jan  7 22:25:40 2017
    gcm@Headless1:~ $ 

    Well, it’s progress…

    Now I can open terminal windows on each worker node for maintenance and / or watching activity with ‘top’…

    Maybe it would be easier to have installed tightvnc afterall…

  7. E.M.Smith says:

    Well, the starting of the ssh daemon / service did not survive a reboot… even putting the ssh file in the boot area didn’t do it (or I didn’t spell ssh right…)


    I really wish people would stop breaking things “to help me”… Another perfectly wonderful tool shot to hell by “helping”…

    I’m packing it in for the night on getting ssh to work AND survive a reboot ( I may just brute force it with a start up script for now…) and go on to something pleasant for the rest of the evening… It’s been a long day and I’m starting to get that edge of grumpy that doesn’t bode well for debugging…

    BTW: The Spousal Mac is dead, Jim…

    The SSD Solid State Disk died. Now just why a stick of memory would die is an interesting question… but it looks like at about the 5 year old mark, you get to pony up $699 or buy a new Mac for $1200… (or there’s a place that sells a kit, MacParts?, for about $300 and you DIY but get more memory…)

    Oh Well…. I’m not going to touch that any more tonight either…

    But at least I’ve got the rest of the config coming along nicely. Ready for a distcc test, more or less…

  8. Larry Ledwick says:

    The SSD Solid State Disk died. Now just why a stick of memory would die is an interesting question… but it looks like at about the 5 year old mark, you get to pony up $699 or buy a new Mac for $1200…

    You running into SSD cell fatigue? We use a lot of SSD’s at work and they tend to die suddenly. The common knowledge belief that they die in a read only mode seems not to be true.

    They just suddenly go toes up and die. Good backups are your friend with SSD and over provisioning them helps extend their life.

    (over provisioning solid state only allocate about 60% of the true drive capacity to file systems so the internal hardware correction system has more spare cells to work with.)

  9. p.g.sharrow says:

    @Larry, Thanks for the links on SSD. Always good to add to my knowledge…pg

  10. E.M.Smith says:

    @P.G. & Larry:

    Yeah… like a $600 mini-SD card of 64 GB or a USB Stick… but on a faster bus… (not seeing the big benefit over 10 Raspberry Pi boards with 80 MB/sec sd cards…)


    Well, I got ssh to work… and fixed a PITA issue with the IP number…

    I’d “done the usual” in the /etc/network/interfaces file. Specified a static IP. Done, right?


    Someone “improved” things some more… I was getting my static IP, but ALSO getting a dhcp address. TWO IPs for the same interface. WhaTh…

    Since Devuan uses the Wheezy type SystemV init, even though this is a Jessie in all other aspects, we look at the Wheezy details. I’ve bolded the issue point:

    Configuring a Static IP for a Raspberry Pi running Raspbian
    The recommend OS for the Raspberry Pi devices is typically Raspbian.
    By default a Raspbian install starts with the nework configured via DHCP. Raspbian is based on Debian, and does things a little differently than their cousins Fedora/RedHat. Instead of configuring resolv.conf directly, it needs to be included with the interfaces configuration file.

    If you are running WHEEZY (older, but still common):

    You’ll need to open the config file for editing.

    sudo nano /etc/network/interfaces

    Change the line with “iface eth0 inet dhcp” and add the lines similar to:

    iface eth0 inet static

    This sets your desired static IP, and the appropriate settings for your resolv.conf You’ll notice it uses your local DNS, a Google DNS (, and one of the Level 3 DNS servers commonly used (

    UPDATE : As of the 2015-05-05 update, you’ll also need to disable the DHCPcd service as well, or you’ll end up with a static and dynamic IP assigned to the eth0 interface. You can disable it with:

    sudo update-rc.d dhcpcd disable

    This will disable dhcpcd from starting on next reboot.
    Arguably, you could do all the static configuration in the dhcpcd.conf file as well, but it seems to be complicating things.

    You should now reboot to test all of your changes.

    So last May someone decided to change how things work and give us two places to configure the same thing, the IP address. Just Damn Stupid, again. Configure one thing in one place one way, please. And stop screwing around with it.

    Per ssh

    After a short foray into the land of iptables (since the “port 22 refused” search term all points at iptables…) I figure out it was the service not staying served… I’m pretty sure this is the magic sauce that got it to stick (as root do):

    update-rc.d ssh enable

    Video / monitor in X

    Oh, I’m going to update the above, but for anyone not rereading above:

    It looks like you can’t get X to find your monitor if you don’t put any new user in the “video” group… that /dev/fb0 file is ‘group video’ and everyone else is blocked in the permissions. So my above chmod 666 was just cludging around whatever security feature they think this provides… So instead of that, just add users to /etc/group in the video group:

  11. E.M.Smith says:

    Oh, and one more from the “everything you knew is wrong again” department…. looks like another gratuitous change was made. I’d adapted to it without recognizing exactly why, but here’s the evidence:

    As Root user restart the SSH service

    service sshd restart

    Please note that on recent Debian systems (e.g. Wheezy 7 with current updates as of Nov. 2015), the above command no longer works and returns the error:

    sudo service sshd restart
    sshd: unrecognized service

    However, the following works:

    sudo service ssh restart
    [ ok ] Restarting OpenBSD Secure Shell server: sshd.

    Note that the difference is the old one started sshD and the new one just ssh.

    In *Nix land, a daemon (program that runs in the background and provides a service) traditionally ends with a ‘d’ for ‘daemon’… now just why you would call a daemon by a name form typically used for a command is a question for another day and another person… Perhaps they combined the ssh command with the sshd daemon into one blob (generally a bad idea…) or perhaps they had typed ssh when it ought to have been sshd one too many time and decided to ‘fix it’ (another bad idea). Who knows…

    I’d just tried starting sshd from decades of habit (i.e. experienced admin) and when it didn’t work tried ssh (a common swap when you type one or the other and it doesn’t work and sometimes happens without even thinking about it as the brain stem doesn’t think about daemon vs not just ‘repeat and delta’…)

    In any case, just FYI, depending on just which Linux you run AND which release of Debian / Devuan, you will have 2 variations of the same command to remember…

    It looks to me like November 2015 somebody really did a number on ssh… Smells like a Poettering thing to me… I hope the Devuan guys get enough time to not only rip out systemd, but find the sequelae and rip them out too…

  12. Pingback: MPICH2 Installed on Pi Cluster | Musings from the Chiefio

Comments are closed.