Raspberry Pi Build Script

Lazy Is Good

I am basically lazy. Lazy is good. It is also known as “efficient”.

I really don’t like to do the same thing three or four times if I can do it once and be done. Consequently, I tend to do things once and put it in a ‘script’ if I think I’m ever going to need it again.

I’ve sunk the better part of 2 or 3 days into figuring out what software I wanted on the R.Pi Model2 to make it a ‘core server’; find out the exact package names to build, and do all the “apt-get install” by hand. At that point, folks usually make a backup image of the product and call it done.

I like having a “build script”. It means your process is repeatable. Done “while doing the build” the first time, it means that if you have a catastrophic failure you don’t have to head scratch about what you did before and / or read notes off of soggy napkins or in cluttered notebooks.

It also means that you can “cooky cutter” another one any time you want.

So here’s my cookie cutter.

The SD Card Software Factory

While the script tells you to do it, this reminder is also a good idea:

You will be running the normal “NOOBS” set up. I’m not going to detail it, as there’s lots of documentation on that already.

Choose to install the Raspbian version of Debian and also check the box in front of the “add a 512 MB EXT file system” option. The script will be marking that partition for NFS export as a nice “scratchpad” you can share with other machines on your network.

When you have all that done, and are logged into the Pi, “become root” before you run the script if you want it to work. You can run it not-as-root, but nothing really gets done. (In fact, I added a bunch of “echo” comments to talk to you while it runs and to ‘syntax check’ it, ran as not-root. Found a few lines that needed close quotes ;-) Hopefully I got all of them…)

I have also taken the pristine “NOOBS download” and put it, along with this build script, onto a 1 GB SD card. I can now use that “portable archive” on darned near any machine to prep the micro-SD card for the Raspberry Pi. Just format the micro-SD to FAT (or FAT32) and copy over the NOOBS files. Then the micro-SD is ready to boot in the Pi as NOOBS install media. I don’t need to hit the ‘NOOBS Download’ every time I want to build a system.

After the basic NOOBS install, I put the 1 GB card into a small USB adapter and plug it into the Raspberry Pi and then copy over this build script. You could just as easily use a USB thumb drive or ‘whatever’.

So with my 1GB “build starter”, and the target micro-SD card, and any computer than can format SD cards and/or copy files, I’m good to run my Raspberry Pi SD card system factory…

The Script

I’m mostly just going to paste the script here. It ought to be more or less self documenting. Any questions, speak up.

I’m also hoping that the WordPress tendency to steal anything with angle brackets in it doesn’t screw things up too much. We’ll see how good my proof reading is ;-)

You will need to scroll the listing to the right to see all of it, but a ‘test mark and copy’ caught it all even off the edge. So a “copy and paste” ought to work fine.

Any line starting with “pi@RaPiM2” is a command prompt from the computer and is followed by the directory in which I am working. In this case, that added 512 MB partition named /media/data/ (but any directory works fine for this.)

Any line starting with a “#” is a comment and does nothing.
It is advisory only.

Any line starting with “echo” gets printed to your screen when the script is run.

The line that says “cat ./BuildIt” is the Unix / Linux way of saying “print this file to the screen” where “cat” means “conCATenate and print” (yes you can list several files and have them all concatenated if desired). The actual script begins on the very next line with echo ” “.

pi@RaPiM2 /media/data $ 
pi@RaPiM2 /media/data $ 
pi@RaPiM2 /media/data $ cat ./BuildIt 
echo " "
echo "Do the NOOBS install: "
echo " "
echo " https://www.raspberrypi.org/help/noobs-setup/ "
echo " "
echo "and choose the option of having a 512 MB EXT partition added to your SD card"
echo "along with the Raspbian installation.  Then copy this script from an external SD"
echo "card or USB drive into your working directory (home directory or /media/data"
echo "after all the usual and customary NOOBS setup questions are answered and done."
echo " "
#
# In general, I'm encapsulating what all I did in these two postings as a script:
#
# https://chiefio.wordpress.com/2015/07/18/raspberry-pi-m2-unboxing-and-setup/
#
# https://chiefio.wordpress.com/2015/07/22/raspberry-pi-software-setup/
#
# If you didn't already change the password while running NOOBS,
# When done, log in as 'pi' password 'raspberry'.  Change the password.
# passwd
# and respond with the new one when prompted.

echo "Also, to change the name of your machine, edit /etc/hostname and make it"
echo "what you like.  "
echo "Here, I'm going to just set mine by brute force write to the file."
echo " "
echo "echo 'Ra2PiM2' > /etc/hostname "
echo " "

echo "Ra2PiM2"> /etc/hostname 

echo " "
echo "Next, do the 'usual' update upgrade that brings you up to the present"
echo "repository status (need a network connection from here on out)"
echo " "
echo "You can either put 'sudo' in front of each of these commands, or just "
echo "'become root' which is what I usually do."
echo " "
echo "sudo bash"
echo " "
echo "then run this script with ./BuildIt (assuming you didn't change the name"
echo "and that you are 'in' the directory where it is located.)"
echo " "
echo "apt-get update"
echo "apt-get upgrade"
echo " "

apt-get update
apt-get upgrade

echo " "
echo "Start doing useful operational 'packages'. "
echo " "

# This gets the useful tools like "nslookup" for looking at Domain Names

echo " "
echo apt-get install dnsutils
echo " "

apt-get install dnsutils

echo " "
echo " VNC is a nice way to get a remote desktop.  It takes some configuring later."
echo " "

echo " " 
echo apt-get install tightvncserver
echo " "

apt-get install tightvncserver

echo " "
echo "I like wicd for an easier way to manage wireless devices and networks."
echo " "

echo " " 
echo apt-get install wicd
echo " "

apt-get install wicd

echo " "
echo "Scrot is a tool for taking screen shots by saying 'scrot' in a terminal"
echo " "

echo " " 
echo apt-get install scrot
echo " "

apt-get install scrot

# Normally I would install "build-essential" to get things like C compiler
# and some language tools, but they were already installed on the R.PiM2.

# apt-get install build-essential

echo " "
echo "Some 'user land' useful things like browser options and Office / Mail tools."
echo " "
echo "Chromium is the 'chrome' browser from Google but in Linux land"
echo " "

echo " " 
echo apt-get install chromium
echo " "

apt-get install chromium

# IceApe is a "more free" version of IceWeasel that is a "more free" version of
# Firefox that is a rebranded Mozilla that is...   IceDove is the matching
# Thunderbird replacement minus the trademarks, non-free bits, etc.

echo " "
echo "Doing IceApe browser and IceDove mail reader"
echo " "

echo " " 
echo apt-get install iceape
echo apt-get install icedove
echo " "

apt-get install iceape

apt-get install icedove

echo " "
echo "GIMP is the photo editor ( 'photoshop Free'...) "
echo " "

echo " " 
echo apt-get install gimp
echo " "

apt-get install gimp

echo " "
echo "Don't forget Libreoffice - Microsoft?  We don't need no steenking MicroSoft..." 
echo " "

echo " " 
echo apt-get install libreoffice
echo " "

apt-get install libreoffice


# I tried "arora" and got error messages and "xbmc" was not working for sound
# so I'm not installing those again until it's clear they work.  Arora is a
# browser (so who needs "yet another browser" with Epiphany in by default and
# with both IceApe and Chromium installed?...) and I'll likely make a dedicated
# SD card for the media center option as there are 2 Pi Model 2 version up
# and I won't need to screw around with Debian issues...
#apt-get install arora
#apt-get install xbmc

# As I also wanted one of these to be a bittorrent server, I sometimes add
# the "transmission" bittorent code.

echo " "
echo "Adding the 'transmission' bit torrent server"
echo " "

echo " " 
echo apt-get install transmission
echo " "

apt-get install transmission

echo " "
echo "To get NTFS disks (like USB or an NTSB formatted SD card in adapter) to "
echo "work 'read write' instead of just 'read only', you need ntfs-3g"
echo " "

echo " " 
echo apt-get install ntfs-3g
echo " "

apt-get install ntfs-3g

# In Theory, this installed 2 VNC "viewers" so the R.Pi could use VNC to 
# get to other machines.  In practice, I found that one of them locked up
# my console when launched against my own machine as target (might be a 
# PIBKAC problem - Problem Is Between Keyboard And Chair - as the R.Pi
# isn't really expecting to drive 2 video sessions at once (the real one
# and the VNC one inside the real one...) so maybe all is fine and I just
# need to RTFM (Read The, er, "Friendly" Manual) before using software...

echo " "
echo "Some VNC Viewers for being the client instead of the server"
echo "I've not used either of these yet so have no clue about them in practice"
echo " "


echo " " 
echo apt-get install xtightvncviewer
echo apt-get install ssvnc
echo " "

apt-get install xtightvncviewer
apt-get install ssvnc

echo " "
echo "Want an NFS (Network File System) server so you can share disks with" 
echo "your internal network?  This will install the code, then you get to" 
echo "configure things like /etc/exports"
echo " "


echo " " 
echo apt-get install nfs-kernel-server
echo " "

apt-get install nfs-kernel-server

# prior to first use.  Or reboot.

# In your /etc/exports file, put something like:

# /etc/exports: the access control list for filesystems which may be exported
#		to NFS clients.  See exports(5).
#
# Example for NFSv2 and NFSv3:
# /srv/homes       hostname1(rw,sync,no_subtree_check) hostname2(ro,sync,no_subtree_check)
#
# Example for NFSv4:
# /srv/nfs4        gss/krb5i(rw,sync,fsid=0,crossmnt,no_subtree_check)
# /srv/nfs4/homes  gss/krb5i(rw,sync,no_subtree_check)
#

# /YourFileSystem  *(rw,sync,fsid=0,no_root_squash)
# But without the # in front of YourFileSystem... and with your file system...

echo " "
echo "IF you chose that 'add a 512 MB partition option' at build time"
echo "This adds it to the /etc/exports file so it is NFS mountable elsewhere"
echo " "
echo "echo '/media/data   *(rw,sync,fsid=0,no_root_squash,no_subtree_check)' >> /etc/exports"
echo " "

echo "/media/data   *(rw,sync,fsid=0,no_root_squash,no_subtree_check)" >> /etc/exports

# Remember to do a 
echo " "
echo "Restarting the appropriate services so NFS will work"
echo " "
echo " " 
echo service rpcbind restart
echo service nfs-kernel-server restart
echo " "

service rpcbind restart
service nfs-kernel-server restart

# I also made my box a static IP number as it's a server.  You will need to
# make this your own server name and IP numbers.
#
# Here's my /etc/network/interfaces file with leading # to make it comments.
# 
# I will make this a "dump these lines in to replace" in my running version.
#

echo " "
echo "Remember to make your /etc/network/interfaces file have a static IP#"
echo "If you are going to be using PXE boot and such"
echo "My examples are below, but use your own values for your equipment."
echo " "

#auto lo
#iface lo inet loopback

#auto eth0
#allow-hotplug eth0
#iface eth0 inet static
#address 172.22.22.253
#netmask 255.255.255.0
#gateway 172.22.22.254
#dns-domain chiefio.home
#dns-nameservers 172.22.22.254 192.168.1.254 192.168.1.1
#
#auto wlan0
#allow-hotplug wlan0
#iface wlan0 inet manual
#wpa-conf /etc/wpa_supplicant/wpa_supplicant.conf
#
#auto wlan1
#allow-hotplug wlan1
#iface wlan1 inet manual
#wpa-conf /etc/wpa_supplicant/wpa_supplicant.conf

# Don't forget to do a
# ifdown eth0
# wait a minute for it to quiet down
# ifup eth0

# As I want this to be a DNS server, DHCP server, and PXE server (uses a 
# tftp or "Trivial File Transfer Protocol" server, all of those can come in
# one package with dnsmasq.

echo " "
echo "Installing a light weight but effective DNS, DHCP and TFTP service"
echo " "

echo " " 
echo apt-get install dnsmasq 
echo " "

apt-get install dnsmasq 

echo " "
echo "Yes, it takes configuring.  See the file at"
echo " /etc/dnsmasq.conf"
echo " "

# Then I installed the Apache web server :

# http://www.raspipress.com/2012/09/tutorial-install-apache-php-and-mysql-on-raspberry-pi/

echo " " 
echo "Instlling the Apache Web Servier and related stuff"
echo " "

echo " " 
echo apt-get install apache2 apache2-utils apache2-doc
echo " "

apt-get install apache2 apache2-utils apache2-doc

# and yes, it takes some configuring and even web page building.
# See files in places like /etc/apache2/sites-available and more.

echo " " 
echo apt-get install libapache2-mod-php5 php5 php-pear php5-xcache
echo " "

apt-get install libapache2-mod-php5 php5 php-pear php5-xcache

echo " " 
echo apt-get install php5-mysql
echo " "

apt-get install php5-mysql

echo " " 
echo apt-get install mysql-server mysql-client
echo " "

apt-get install mysql-server mysql-client

#
echo " "
echo "And that's the end of my present install build process."
echo " "
#
# There are several files to edit and configure.  Eventually I'll add a 
# "here script" to dump them from this script to where they belong, or 
# I'll just save a copy and have a 'save / restore' copy process.
#
# Once I get everything configured ;-)
pi@RaPiM2 /media/data $ 

With a fresh 64 GB micro-SD card, it took me about 30 minutes to do the basic NOOBS install and copy files. Then it was another hour to run this script (almost to the minute). That will vary with your network speed and how diligent you are about hitting “Y” when prompted. ;-)

I’m posting this using that chip and the IceApe it installed, so things look to have worked reasonably well.

One quirk I ran into: The NOOBS install did NOT like being behind two layers of NAT (Network Address Translation). I had to move from that 172.x.x.x network out to the 198.162.x.x network to get it to work. I’ve run into that kind of issue before with other products, so it isn’t a Raspberry Pi issue. Just be aware that NAT screws around with network addresses and that some things don’t like them changing underfoot when in use. OTOH, it can sure confuse folks trying to break in ;-)

I generally avoid it, but the router I had available for fast use was already configured with it and for most things it has not been an issue.

After final configuration of those various config files (like for PXE and NFS and such) those config files will also be copied off to the 1 GB chip. I may eventually incorporate them into my ultimate “build script”, or may just leave them as files to drag over by hand if needed.

As the various services I’ve installed “go production”, I’ll post more specifics on how to set it up. But in many cases, the specifics will be specific to your own site and needs; so mine will be more exemplar than ‘copy paste’.

In Conclusion

By making this script, at any time I’m about 1 to 2 hours away from a fully built system. Even if 6 months from now I forget some bits. Even if I’m on the other side of the country via plane stuck in a hotel room and want to make one. Never again will it take 2 or 3 days of my life to paw through the jungle figuring out what matters and what is hiding where. What is missing in the basic build. As my needs change, or new things are discovered, they can be added to the script. New versions made for other special uses. Each time it is used it is, effectively, regression tested, and over time any small errors get ironed out, never to return. That is the benefit of such a build script. The downside is that capricious upstream changes in packages will take some maintenance, but the script will be shouting error at you when that package is no longer available and you will know that something changed.

Hopefully this will be helpful to others in their:

Happy Hacking!

Subscribe to feed

About E.M.Smith

A technical managerial sort interested in things from Stonehenge to computer science. My present "hot buttons' are the mythology of Climate Change and ancient metrology; but things change...
This entry was posted in Tech Bits and tagged , , , , , , . Bookmark the permalink.

15 Responses to Raspberry Pi Build Script

  1. R. de Haan says:

    Thanks for this and all the other Raspberry posts E. M. I’m pretty sure I will put them to use.

  2. E.M.Smith says:

    @R. de Haan:

    You are most welcome.

    It just makes sense to “split up the work”. So IF I’ve figured something out, might as well put it where other folks can find it and avoid the time loss. Like finding out that “nslookup” is hidden in dnsutils… Sure, only takes a small search and some time digging into things. But a dozen of those starts to be all day…

    Besides, in another 6 months when I get another one this will remind me what I did ;-)

  3. Larry Ledwick says:

    I may have to play with one of those now with that script as a head start. Working in IT I it is hard for me to motivate myself to do the same stuff in my free time. I threw away too much of my life trying to get some of the earlier Linux distributions running. I am even more minimalist than you would only use it for web surfing and checking email.

    Looks like I could comment out the apache stuff, and gimp (I really dislike their interface), and probably VNC, and wicd since I do not use wireless networks, torrent and php/mysql.

    Nice thing about a script like that you can comment stuff out and try things and see if you lose any functionality you care about.

    Nice!

  4. E.M.Smith says:

    @Larry:

    Thanks!

    Yeah, all the parts are independent. If you comment out GIMP, all you lose is gimp photo editing. Comment out Apache, you don’t get a web server.

    The other nice thing is that the way apt-get works, if there IS a dependency on a package you commented out, those bits will be selected for adding back in with the other package you left in the script.

    FWIW, Linux is now, usually, far better than in the early days. Even a “young” port is usually mostly functional right out the gate. Software installs and such are now nearly trivial in comparison. ( “apt-get install build-essential” to get all the typical compilers and tools? And even that is already done in the R.Pi build?)

    You can see where I commented it out (since it just says “already got it chief”….) and also the two that “need work” of arora and xbmc. Easy peasy.

    Yes, I’m “pushing the edges” as part of what I want to know is “Can this be a reasonable desktop?” and the answer is “Yes, sort of” with a bit of sloth sometimes (but not enough to matter to me) and with a couple of things that don’t work (but since this is a new card, not a surprise and likely to be fixed in a few months).

    For general email, web browsing, and “office” things, it’s dandy. IMHO.

    That I can put it in my luggage on trips and not care if it gets lost, or if someone thinks they can clone it they find the chip is in my shaving cream can lid and not in their back room… If there’s a “knock on the door” that micro-SD card pops out in 1/2 second or less and will disappear into just about anywhere (including between tooth and cheek…) all while my data can be on an encrypted disk or even sitting in a inscruitible binary blob in a ‘cloud’ somewhere. What’s not to like?

    That most of the time it will be doing mundane file server and DNS service with the odd PXE server handoff of an OS to the old EVO is just something to keep it busy so it earns its keep ;-)

  5. Larry Ledwick says:

    I like the idea as an emergency backup computer that is cheap enough I can toss it in an old ammo can along with a few bits like a usb drive, etc. and have a spare system that you can get by with even if the house gets struck by lightning and fries everything that is plugged in, or some other disaster happens and is cheap enough you won’t mind that it is just sitting in a box.

    I had the lightning strike years ago, hit the HF antenna on my shortwave rig and killed everything plugged into that circuit until the surge got to the power panel and found good ground. Blew the surge protector out of the wall socket, but I got lucky the surge came down the ground side of the antenna and went straight across the back plain of the radio and caused no damage to the radio at all ( still have it still works — it was turned off at the time though).

    It blew the tops off of the chips on the computer mother board in the living room which was plugged into the same power circuit.

  6. LG says:

    Danke ! Herr ChiefIO.
    Danke Schön.

  7. p.g.sharrow says:

    @EMSmith; I like your build script. Surprise! I think I even understand most of it. Comments help a great deal. We will soon see if I can get this to work as I have most of the needed parts gathered. Still need a dedicated display. These old SVGA CRTs are reaching the end of their life. I will be upgrading from this old Celeron XP, so Raspbian/Linux will be new world for me. This machine and our shared satellite connection is too slow and flaky to do this from here so I will have to do the download from the shop high speed connection with one of the much newer computers. Every thing we have has been Windoz for 30 years. Command line control will be “back to the future” of DOS-3 days for me 8-l. We will see if this old dog can learn new tricks. pg

  8. E.M.Smith says:

    @LG:

    You are most welcome.

    @Larry:

    I have 3 computers that are not connected to any wires as “backups”. Generally when I deprecate a system, it just gets set aside intact. For “emergencies” (or for anything I screwed up in the transition…) With the low cost, no fan noise, etc. etc. of the RPi, I’m likely doing to change that to “board in a can with a chip”. The 10 GB and 40 GB disks on my two old Linux reserve machines that seemed so big, then, are now a fraction of a micro-SD card. The AMD 400 MHz PII class processor is now weak in comparison to one of these 4 core boards. So I sense a shift coming in how I do “archival preparedness boxes”.

    Like you pointed out, one of these, a USB backup drive, and a couple of system chips all fit in a SMALL ammo can and packed in antistatic bags and bubble wrap would live though anything short of a nuclear bomb. Toss in a dinky 100 W inverter and you can run from any car battery for a few months ;-)

    @P.G:

    That’s what comments are for. Life has a way of teaching you things. For me, it was running into a chunk of code. It was well written. A bit obscure in what it did, but that programmer had left some nice comments in the code explaining some of the more ‘trick’ bits. I managed to figure it out and was impressed with some of the more elegant bits. Without the comments it would have been a royal PITA to figure out.

    But it was my code from a decade or so prior… It changes you when you first realize that the “future maintenance programmer” looking back is you looking in a mirror…

    @All:

    Not all of you are Linux / Unix script writers. I’ve done it for decades and the language is roughly the same between Bourne Shell (sh), and bash (Bourne Again Shell) and even KSH the Korn Shell. For those wondering, Bourne and Korn are really names of real folks who wrote the code for those shells or command line environments. It is WELL worth learning. The shell is a “threaded interpreted language” and you can easily build up a library of “scripts” to do all sorts of things. With that said, about the script above:

    A couple of clarifying notes:

    I said anything starting with “echo” prints to the screen. That isn’t quite true. It prints; but you can redirect the print to somewhere else with an angle bracket, so:

    echo “Ra2PiM2”> /etc/hostname

    sends the text Ra2PiM2 into the file /etc/hostname

    As there is just one angle bracket, that says “empty the file and THEN put this into it”.

    After a few decades such “tricks” in Unix / Linux land become part of your normal thinking and one tends to forget that it isn’t obvious to folks from MS Land… but that’s what the angle bracket means in that context. Later I do the same thing but with two angle brackets in a row. That means “don’t erase what is already in the file like one angle bracket does, just append at the end”. That lets me write the one added line to /etc/exports that lets the /media/data partition be exported; without erasing all the rest that is in the file.

    echo “/media/data *(rw,sync,fsid=0,no_root_squash,no_subtree_check)” >> /etc/exports

    The default location where that 512 MB partition is mounted is named /media/data so ought to stay the same from build to build. At the far other end you see the two angle brackets that say “tack this onto the end of /etc/exports but don’t erase anything already in the file”. In between is the set of NFS attributes for that particular export of a file system. In particular, note the *

    That says “export it to anyone at any IP address anywhere”. Fine as a “likely to work” on my private test network; but not particularly security oriented for private production. Usually after NFS is shown to work, I’ll prune that back to the minimal permissions. So I’d replace it with 172.22.22.0 to limit the machines that can mount that file system to “just my lab address block” ( no sense putting it out where the spouse and her Mac can see it, as it’s just my tech stuff…) Also note that there is no space between the * and the open parenthesis. Put a space there, it breaks (at least on some unix / linux releases). The “rw” is “read write”. You can export “ro” for “read only”. That is what my OS copies will eventually be when exported. So the Evo can boot from a copy, but no one can write back to the data store. Any “hack” of that OS running on the EVO dies when the system reboots. “no_root_squash” says to let me BE root on that file system. This lets me (on some other box) be super-user even over things on that NFS mounted data. Nice for me, but a security risk in production. Normally root is “squashed” and gets no privileges. You will likely want to take that out once things are shown to be working.

    Also note the general repeated structure of the script. Tell ’em what you are going to do, tell ’em what you are doing, do it.

    # As I also wanted one of these to be a bittorrent server, I sometimes add
    # the “transmission” bittorent code.

    echo ” ”
    echo “Adding the ‘transmission’ bit torrent server”
    echo ” ”

    echo ” ”
    echo apt-get install transmission
    echo ” ”

    apt-get install transmission

    The first lines, that start with #, are comments. You will not see them when the script is run, only if you edit or read the script itself. They are notes to some future user or programmer doing maintenance on the script. Often these are redundant with the advisory messages being printed out at run time, but “I have a habit”… so you get both.

    In this case, the comments say “going to install a bittorrent server”. You could change which one by changing the actual install command to a different one, and this comment would stay valid.

    Then I have an “echo” that prints on the screen at run time that the script is installing a particular bittorrent server. It can be very useful when something is running for an hour, and you are wandering back and forth checking on it, for it to make a clear block of space with an advisory in it saying “Doing THIS now!”…

    Then, the only part of that whole block of text that actually does anything in the install process is:
    apt-get install transmission

    everything else is there so that in some future time some future “me” doesn’t need to remember all this, nor will future-me be wondering what that script is doing 1/2 way through a run, nor why I didn’t install arora browser ‘back then’. I also include some examples of the config files that need changing “as comments” to remind me what to do after the code is run. “Someday” that will be scripted as well, but for now, it’s just reminders to “future me” not to forget to do the config file changes.

    The actual “meat” of the script is about 2 dozen lines. The rest is directed at people.

    At some point I’ll get a real automated “build system” built. Something like “chef” or “puppet” or similar. Their whole job is to consistently build a system from “scratch” based on a stored recipe. But for many uses, a simple build script is enough. Besides, I’ve been doing this since before such build systems were in existence and both it, and the habit, work… so I’ve had little reason to ‘move on’.

  9. Larry Ledwick says:

    How do you feel about at the end of your build script doing something like this for your major files like the /etc/hosts /etc/fstab etc. That would in effect on install capture all the configs for a clean install and if run as a cron periodically would automatically backup your tweaks as you lock down and expand the system.

    cat really_important_config_file > really_important_config_file_bkup.datetimestamp

    At Sun they had a script that ran on a cron script that captured key system config file images and status commands such as df -k output, the fstab file, routing tables, crontab scripts themselves etc. so if they got clobbered they just needed to copy the most recent image back over the clobbered file and you were back up and running in the time it took to copy the files.
    (crontab script is a script that is run by the system at a scheduled time every day, week etc. often for maintenance)

    It was a life saver on a couple of occasions when either files got accidentally stepped on or non-persistent changes were made and when the system rebooted it forgot the changes that folks forgot to document.

  10. E.M.Smith says:

    @Larry:

    Strange you should mention that…

    Not in this posting was that I copied all the “usual suspects” to my 1 GB “creator” SD card. So I did a set of:

    cp /etc/passwd /USBpath/etc_passwd
    cp /var/www /USBpath/var_www
    cp /etc/network/interfaces /USBpath/etc_network_interfaces
    etc.

    Yeah, long hand. On my “todo list” is to make the copy scripted, and the copy back as part of the system rebuild automated. But since I’d not finished all the configs yet, figured a ‘one off’ was fine for now. I have to get the PXE server configured and working and the web server set up (test page worked!) and more. Then it will make sense to have dups.

    FWIW, due to some “unfortunate learning experiences”, I make one manual copy that lives on the “re-install media” from the base case, and any automated copy rotations go somewhere else. That way if you have a script copy over your last good copy with the broken one, you still have a ‘first good copy’ on other media…

    (Nothing like finding out that your 7 days of rotating archives has been carefully updated for 8 days after that script got broken before anyone noticed… )

    And yes, using “cat” as you did also works and I’ve done it… but “cp” is the tool designed for it and takes 2 characters less ;-)

    (Unix folks HATE typing even one character too many… so cp is one shorter than cat and it doesn’t need the redirect angle bracket… 2 chars saved. Do that a million times in your life you are talking weeks of time saved not to mention wear on the fingers ;-)

    On systems I’ve managed, we had all sorts of automated things running. Doubt I’ll put most of them up on the Pi. Things like hidden scripts that compared byte counts on system binaries and modification dates on key files and if anything changed would put a warning on the sysadmins console to go check for hacking… Now I’d likely use a HASH instead of a byte count. Or more likely both ;-) Things that would “T” the syslog off to a remote location so that the syslog could NOT be erased or changed. (Caught a hack attempt that way once… the syslog printed on a line printer inside the locked computer room – system was “Apple.com” a VAX in another bulding and our “honey pot” in addition to open system for apple family users. Operator noticed something on the printed log while inspecting / changing paper… called sysadmin… hack caught “in progress” when sysadmin found online syslog had those lines deleted… Paper in a remote locked room is your friend ;-)

    We also had “run books” for each system with key text files printed and stored in it for reference and ultimate recovery. Yeah, a bit over the top vs tech now.

    But yes, a couple of copies of key config files is an important thing to have. Done by hand, automated, or both. My eventual final build script (once all is configured and running) will have an automated “copy in these files” from the USB dongle disk and I’ll likely have a symmetrical “duplicate this system config out to BUP USB” script… someday… after I’ve got it doing what I want… If I have enough time… and beer ;-)

  11. E.M.Smith says:

    The above was not enough to get a real working transmission. Needed to add the daemon:

    root@RaPiM2:/home/pi# apt-get install transmission-deamon
    Reading package lists… Done
    Building dependency tree
    Reading state information… Done
    E: Unable to locate package transmission-deamon
    root@RaPiM2:/home/pi/tails/EMS# apt-get install transmission-daemon
    Reading package lists… Done
    Building dependency tree
    Reading state information… Done
    The following extra packages will be installed:
    transmission-cli
    The following NEW packages will be installed:
    transmission-cli transmission-daemon
    0 upgraded, 2 newly installed, 0 to remove and 0 not upgraded.
    Need to get 1,253 kB of archives.
    After this operation, 2,485 kB of additional disk space will be used.
    Do you want to continue [Y/n]? y
    Get:1 http://mirrordirector.raspbian.org/raspbian/ wheezy/main transmission-daemon armhf 2.52-3+nmu2 [212 kB]
    Get:2 http://mirrordirector.raspbian.org/raspbian/ wheezy/main transmission-cli armhf 2.52-3+nmu2 [1,041 kB]
    Fetched 1,253 kB in 14s (87.2 kB/s)
    Selecting previously unselected package transmission-daemon.
    (Reading database … 89934 files and directories currently installed.)
    Unpacking transmission-daemon (from …/transmission-daemon_2.52-3+nmu2_armhf.deb) …
    Selecting previously unselected package transmission-cli.
    Unpacking transmission-cli (from …/transmission-cli_2.52-3+nmu2_armhf.deb) …
    Processing triggers for man-db …
    Setting up transmission-daemon (2.52-3+nmu2) …
    [ ok ] Starting bittorrent daemon: transmission-daemon.
    Setting up transmission-cli (2.52-3+nmu2) …

  12. Pingback: Do NOT install SliTaz .deb on Raspberry Pi Model 2 at this time | Musings from the Chiefio

  13. Pingback: Well That Was Fun, sort of… | Musings from the Chiefio

  14. E.M.Smith says:

    in playing with file systems I found that I needed to add:

    apt-get install btrfs-tools

    Note that
    apt-get install xfs

    installs the X-font server and not the xfs file ssytem.for that you need:

    apt-get install xfsprogs

    While for ntfs disks do:

    apt-get install ntfs-3g

    gives you NTFS file system support

    and for Macintosh file systems:

    apt-get install hfsplus hfsutils

    At present, f2fs isn’t building for me, I think I need the newest kernel, but someday:
    sudo apt-get install f2fs-tools gparted

    To get both f2fs the Flash Friendly File System and the gparted partition editor (that did build for me).

    I also wanted to try squashfs for read only compressed images:

    apt-get install squashfs-tools

    And, for UnionFS in User Space:

    apt-get install unionfs-fuse

    It also looks like AUFS is already in there somewhere but maybe not loaded / active?

    root@dnsTorrent:/home/pi# modprobe aufs
    root@dnsTorrent:/home/pi# zgrep -i aufs /proc/config.gz 
    CONFIG_LOCALVERSION="-rpi-aufs"
    CONFIG_AUFS_FS=y
    CONFIG_AUFS_BRANCH_MAX_127=y
    # CONFIG_AUFS_BRANCH_MAX_511 is not set
    # CONFIG_AUFS_BRANCH_MAX_1023 is not set
    # CONFIG_AUFS_BRANCH_MAX_32767 is not set
    CONFIG_AUFS_SBILIST=y
    # CONFIG_AUFS_HNOTIFY is not set
    # CONFIG_AUFS_EXPORT is not set
    # CONFIG_AUFS_RDU is not set
    # CONFIG_AUFS_PROC_MAP is not set
    # CONFIG_AUFS_SP_IATTR is not set
    # CONFIG_AUFS_SHWH is not set
    # CONFIG_AUFS_BR_RAMFS is not set
    # CONFIG_AUFS_BR_FUSE is not set
    CONFIG_AUFS_BDEV_LOOP=y
    # CONFIG_AUFS_DEBUG is not set
    

    So I guess I need to figure out if it is already live or if I need to do something to wake it. I think that 2nd line with AUFS FS=y means it is already alive in the kernel…

  15. Pingback: RaTails – Draft High Level Steps | Musings from the Chiefio

Comments are closed.