The purpose of this posting is to lay out the high level steps I see in making an analogue of Tails on the Raspberry Pi.
Partly this is a working document for the project of making something relatively secure and disposable that is ‘sort of like Tails’, and partly it is a set of links to things along that line that are interesting.
Any suggestions and / or folks pitching in most welcome.
Why Do This?
I’ve used my Raspberry Pi boards for several things now, and the Raspberry Pi Model 2 is “comfortable enough” for use as a disposable workstation (though you might want a bit more eventually). Single core (CPU) performance is a bit low for high demand tasks (like WordPress article editing – but some of that is network load as WP is very “chatty” with spell checkers constantly resending all the text). Multi-core performance ‘needs work’ on spreading task load around, but that’s not a big problem for most folks.
That means that you are $60 to Amazon and 1 day of assembly away from restoration of any “lost” (i.e. stolen by over zealous police) hardware. For typical uses, you could also have a stash of pre-built SD cards and just stick one in and go. Or download the image from some remote site, copy it to the card (with Windows or something like dd on Linux), and go. But then you end up in a recursion problem. You need to make the workstation to get the image to build the workstation…
And, if your ‘stuff’ was just taken (rather like what happened to Tallbloke ) and you know you are under surveillance (even if misguided) and perhaps with active Man In The Middle attacks on your downloads… well, you need some ‘other way’ to have internet presence without the probability of being compromised.
Progress To Date
Earlier we looked at how Tails is built and what if anything was being done by the Tails folks on the Raspberry Pi. First off, it’s a complicated build using at least 3 languages and git just to do a ‘roll your own’ OS. That’s a steeper curve than I can get done in short order and it is not well suited to a “newby” learning how to do things on a R.Pi. (That education goal being part of the whole reason for the Pi…)
While I’d love to just have a link to “install Tails on your Pi from here”, that isn’t in existence yet. The work to port Tails is larger than one part time guy wants to take on (i.e. me) and I think an incremental approach is more productive faster (though eventually a port of real Tails will obsolete it, the skill sets will be valuable in any case). So I also looked into alternatives. More on that below. But what has been done?
The basic hardware build and software configuration steps are documented and a draft generic build script written. While investigating precursor tech, like a ‘Live-CD’ build, I found that SliTaz has a ‘Live-CD’ like build for the R.Pi, but it isn’t ready for the Model 2 just yet. That could serve as a base for building a RaTails on the R.Pi B+ model, but don’t put it on your Model 2 without testing on a recyclable SD card… We have also seen how to put a file system into an encrypted file container on the Pi.
All of which brings us more or less up to date. Raspberry Pi built, configured, running and with the ability to put things into an encrypted container. Knowing that a ‘Live-CD’ build is possible, and that TOR The Onion Router, is already ported to the Raspberry Pi and running.
What’s Next?
This is a very high level list of what I see as ‘the next steps’. There are some dependencies in these, but often they can be done ‘out of order’. So some of this ordering represents personal interest, what I would want to have working first, or just random listing order. Feel free to go out of order. This isn’t a PERT chart with a critical path and dependencies, just a ‘ToDo List’.
1) Make a version of the OnionPi. I’ve already made a Pi that uses a WiFi dongle so this is mostly adding the Onion Router code to it. I put this first simply because it is ‘ready to go’ and will be immediately usable “as is”.
2) Boot it from an external USB drive. At that point you can have your “goods” on a ‘yankable’ USB stick and a very generic boot SD card in the Pi itself. This also lets you set up for boot-from network (and eventually from a remote site if you like). Berryboot looks like the way to go and doesn’t look too hard. This step could be done first, if desired.
3) Make that a ‘Live-CD’ type boot. This step starts to be a bit harder. I noticed that the Berryboot link shows Puppy as one of the OS types. As Puppy is a ‘live-CD’ boot, it might make sense to use it as the base and put the TOR router in place. That is an investigation step. “Roll your own” live-cd’ or start from pick one: {SliTaz | Puppy | …}. This step could be done first too, if desired. This involves using a read only squashfs file system to store the OS, and having a RAM based writable file system overlay using AUFS or similar on top (union file system) so it looks writable, but everything is really in a ram disk. It also means Raspian is out as it sometimes rolls to swap even with a 1 GB memory. Take 1/2 that for a RAM disk and you will ‘have issues’. So a smaller lighter distribution…
4) Add encryption to the boot. The Raspberry Pi has a rather different boot process. At some point it would be desirable to have the root / boot process be from encrypted disks, but it is a potential performance issue and complicated. I’d leave this for a later step; though Kali looks to have done it. It would be good to have a pen-testing station in your site anyway, so building one of these is a good idea in any case and in any order.
5) Amnesiac Build: Go through the system and weed out all the places it tries to remember things and squash them. At this point, reference to the Tails build is likely most efficient as they have done the work already. Things like making the browser not save cookies and not save cache pages or history. Can be done in any order, but likely better if done after base OS is settled in step 3 and many writes go to RAMdisk so volatile anyway.
6) Incognito Enforced Build: Go through the system and weed out all the places it tries to identify you and squash them. At this point, reference to the Tails build is likely most efficient as they have done the work already. Things like making the browser not tattle on you and not reporting things like all the WiFi stations your dongle can see and doesn’t present any useful identifiers of release info. Can be done in any order, but likely better if done after base OS is settled in step 3.
7) Harden the Operating System and Tools: If you don’t NEED it on the device to function, take it out. What is left needs to be tested and assured hardened. Firewall and protective suite tools tuned. So things like proper use of encryption, certs, key lengths, ciphers, ACLs etc. Use Kali for a first pass test of it. Can be started in any order, but must be completed after the system is stable and more or less finished. Create a build list that only includes the known needed elements and has ‘remove’ for everything else. Any specially modified code needs a repository built {somewhere} with source available for scrutiny. Ensure there are applications and pointers to things like encrypted email services, key generation and management, VPN to other countries, etc. All the applications you need, nothing that isn’t essential or secure.
8) QA and stress test cycle. Build a regression test suite for all known issues. (Perhaps start from the list of historical issues encountered by other releases such as Tails and Kali.). Ought to start collecting issues and data from day one, but this step never finishes as it is re-run and expanded with every new release and bug report.
9) Beta tester release. “In the wild” trials and fix cycle.
In Conclusion
Well, that’s what I’ve got so far. Kick it around. I’m quite happy to expand, contract, fix, and update.
My best guess is that I’ll get through about #5 and the real Tails release will get ported. Still, I think it worth the effort. There are times I’d like something a little more custom and flexible (so, for example, I can choose to save some cookies and and I can have a few desired but not quite mil-spec tools available) and Tails is not going to let me. Or I’d like a locked down amnesiac system, but don’t really need a full on TOR slow routing experience.
So IMHO being able to ‘pick and choose’ makes this worth it. It also means I’m not depending on the purity of some other soul or some download site for what I get. There is value in being different sometimes. For example, if “agencies” are watching for the fingerprint of folks using Tails or camping on TOR exit nodes, being a bit out of spec is a feature. (“Rotating shield frequencies Captain!”…) It can also “finger you” if you are the only person with that profile, and it means you are responsible for finding all the exposures in your ‘kit’, but having both modes available IMHO has merit.
I have a couple of other links on the tablet that need to be reviewed and I might add some more here later. Also note that each step needs a WBS Work Breakdown Structure done to sub-tasks. For example, a ‘live-CD’ type read-only build depends on having a RAMdisk based OS and a squashfs archive, though for a larger image on a USB drive, the squashfs is optional. Incognito Build depends on having an incognito hardened browser ala Tails. Each of those sub-tasks needs to be identified and then done. For a One Guy Shop that’s easy. You just do it and fix the errors, omissions, and aw shits in the QA cycle. For ‘more hands’ they need a bit more listing and handing off.
All of which says “expect things to grow in the comments” as that detail surfaces. Feel free to surface it ;-)
At this point I’ll watch and listen as I fill my parts bin. After you get the bugs figured out and things streamlined I’ll do the software things. Good thing a Pi “brain transplant” will be a simple SD chip change.
and USB “plug in”. ..pg
Jumping waaaay out ahead:
http://elinux.org/RPi_iSCSI_Initiator
Has a very nice write up, including Model 2, for making a Pi boot from a net attached iSCSI server (that can also be a Pi with USB storage). Post boot the SD card can be removed. At that point the Pi is a workstation with zero storage on it, so zero saved and zero function if you powerfail it. Pull the plug or hit a toe switch, it goes very dark. Nice.
Yes the OS and any saved user space is on the iSCSI server, but that can be miles away, or in a closet upstairs also powerfailed… and needing a decryption key on boot anyway . Take the hardware, you get a naked Pi that has no OS on it, a usb kb & mouse, and some monitor… nothing more.
@P.G.: Remember, they also serve who only laugh and point… :-)
At times I do point————> no laugh, but often smile;-) …pg
Just call it “defect discovery and QA” …. a valuable technical contribution :-)
I’m ashamed to say I didn’t know about Berryboot. But looking at the site, it looks like, as you say, it might be the best first step. I have the Raspberry Pi that I set up for the Onion test, so I’ll try it out on that and report back.
I would suppose, depending on how much Berryboot loads onto the first partition and what *it* logs will decide whether that should be the first step. If the answer is not much and not much, then that definitely has to be the way to go. It would definitely be the way I would go with cubie.
Do you know if it’s possible to rsync into a SquashFS filesystem? What I’m thinking is for backing up an OS, you could have a really nice recursion. So OSorig backs all its new stuff up onto OSBackup. Berryboot boots up OSOrig, but it doesn’t load so Berryboot uses OSBackup, which now becomes OSOrig which rsyncs itself to OSBackup. You would never even know your first OS died.
@Paul:
No reason for shame… I have Berry Boot on my first R.Pi but it is headless and set up to boot to a specific image anyway and automatically… so knew all about it a few years ago; BUT… only remembered that when I saw the BerryBoot images on their website and thought “Oh, yeah, I did that some years back…” One of those forehead facepalms where you think “I knew that, really I did…”
Seems to me that’s more of a reason for self shaming than just not having seen it in the first place…
Per SquashFS: I did some investigation into that. I’m about to do some experiments with it and get the skill from the frontal lobes into the brain stem and fingers ;-)
The process looks like a particular tool is used to “squash” a real filesystem into a squashed image, and that image is just one big compressed file, so not ordinarily writable. HOWEVER, since it is just a large compressed lump, nothing prevents your cycle where you just have a backup copy of the lump in case you munge the working copy. However, however, you would need to make changes in a live non-Squash FS in order to generate a new form of SquashFS so it would be easier to keep copies of the ‘source FS’ with deltas…
http://unix.stackexchange.com/questions/18777/how-to-construct-a-read-only-compressed-modular-filesystem
was pretty clear and the model I’m going to follow.
My intent is just to take a generic R.Pi image, do a mksquashfs on a few key directories of things like binaries that ought not to change, and mount them as RO Squashfs and see how well it works. (And adds a layer of ‘protection’ in that the binaries can’t be updated as the FS is mounted RO but also as the underlaying is a compressed wad that can’t be re-written easily over the wire while live…
Then, next step, being to make a writable overlay for some bit (likely something like /tmp just to see if it works) and test that whole AUFS layer. Only then going to the SquashFS RO underlayment and the overlay of RAMdisk AUFS. Say for /var …
At that point, you are set to have a base OS copy RO, and still make changes via an AUFS layer, but for persistence would need to do some kind of copy out / merge or copy out / reload (those persistence files in some releases… but I’m not there yet…)
It all looks pretty simple and very straight forward (once you have the tools installed)
So for your circular copy to work, you would need to do a
mksquashfs /modifiedfs NewOrig.sqsh
then if the boot failed, point to OldOrig.sqsh instead…
and if the boot worked, copy NewOrig.sqsh OldOrig.sqsh…
Or some such
I expect mksquashfs can run against a layered AUFS mounted file system image and suck up both changes AND the underlaying uncompressed squashfs sourced files and roll the combined image back into a new squashfs image… No idea how fast or any other tuning / issues needed. (Thus the planned experiments… once my R.Pi isn’t downloading data… which it is STILL doing after a week… Up to about 200 GB now… sheesh… )
Looks like there is an ‘append’ option on mksquashfs:
Given that there is no “append now” command, I’d guess that by default it will append to an existing squash file and you must tell it not to… and to overwrite instead with -noappend.
For “someday”: that filter setting to know the type of CPU you are on is likely very helpful for compressing binary compiled files… Also note there is an option to sort the contents in preferred order, so once you get into performance tuning then things can be grouped by speed to access.
Easy to use with defaults, but some thought needed to optimize..
Nice and fast:
so about 1/2 the size. I also note that it is aware of all 4 processors and used them.
Now this was all just way too easy… Here’s the /etc/fstab entry I made:
after mkdir /usr/sqsbin and then I just did the usual “mount /usr/sqsbin” and:
So, should I desire it, I can take just about any static directory structure in the whole system, darned fast make it a .sqsh blob, and then mount that instead. ( I’ll have to find out what the loop device limit is, given all the things I’m mounting as loop file systems… I think it is about 7?)
Since you can mount one file system on top of another, the squashed version ought to be OK to mount over the original. Use “no-auto” in the fstab entry, and if the mount causes a crash, on reboot it will not be mounted and all will be normal. Darned easy way to protect the original binaries from corruption and / or hack attack…
So let’s try that… {time passes} and it works fine:
Golly, I think I like this…
Another Update: Well, I did it with all of /usr and I’m running on it now:
All well and good… but…
I thought “Hey, nice, I think I’ll unmount it and move it to a USB disk and run from there… and promptly was reminded you can’t unmount a file system that is in use even if there is a duplicate under it… and that /usr gets a lot of use.
So now I’m committed to it until such time as I next shut down and processes wrap up use of /usr. OK, now I’m very glad I had ‘noauto’ in that list of args… (The “fix” were it not specified, would be to remove the mount entry in /etc/fstab and reboot again. That would let the unmount proceed during shutdown, but not do a mount during the reboot.)
At any rate, things are running fine, so looks like “no worries” so far. I’m going to let it run a while (days?) as I’ve got some background tasks running and just figure this for a QA run. I’ve already made a copy on to the external USB (as the blob.sqsh is just a file… even if the contents are a mounted file system I can still read / copy the blob) and I’ll do that exercise another day, or on another filesystem.
I can see one easy indirection being to have a boot from a generic SD card, then mount a remote or USB based /etc on top of generic /etc and have that /etc/fstab do all the other remote or USB mounts. Then, on a power fail or shutdown or just ‘awshit’, you can pull the usb drive and book it. Anyone booting the Pi just gets a nice generic OS with no evidence it was actually running a different set of files from a removed drive… A nice “first step” toward a RaTails.
Still would need things like the Logs and /var sanitized to be really “clean”, but that’s on the “todo” list… (easy enough to have them be non-squash so writable but mounted from the removable device.)
Making a RAM disk is also easy, but one ‘bug’ had me for a while. First off, there are 2 kinds of ram disk.
Type ramfs does not show up in a ‘df’ listing and can roll out onto swap and grows automatically to whatever size is needed (and in theory could use all memory and crash things).
Type tmpfs has a fixed size (with small default) and does show up in a df listing.
Both are made via the ‘mount’ command, but the syntax is slightly different.
This first form makes a ramfs type space (starts at size option grows unlimited). We will make a directory, then mount some RAMdisk at that point in the name space
doing a df shown nothing, so I won’t post that here… Simply umounting /mnt/ramdisk frees the memory.
This second type makes something that shows up in a df command and is of fixed max size:
For easons I can’t explain the -o and -t had to be ‘backwards’ in this one. Trying it the other way round gave error messages… and yes, it took a while to figure that one out as the man pages tells lies about it… “Ramdisk” is just a display name that shows up in the df listing and can be anything you want. (For a while I was seeing “size=512” as the device name… that was my clue that something was wrong … then I added ‘Ramdisk’ and only THEN did it fail as it was complaining about the size= parameter… that led to the discovery about order… which the man page says ought to be -t then -o but that doesn’t work in this form…)
Here’s a partial df listing and you can see it worked and shows up:
It didn’t show any change to memory, so I had to make a file in it to see changes. Here’s the making a 100 MB junk file, removing it, and the memory changes in between. Notice the first entry under the ‘shared’ heading in the ‘free’ listing:
Next I need to figure out how to ‘layer’ this over a RO squashfs and then we can have our read-on ‘Live-cd’ like structure.
Hi Chiefio,
Wow, thanks for doing all that. That’s amazing work. The first option, writing the filesystem to a temporary file, and then squashing it and using it to overwrite the previous backup would give me much of what I need. The only thing left is figuring out where to put the conditional – if this os doesn’t load use this other one. See below.
The Berryboot install went great. Download it, copy it over to the SD Card, put it into the Pi, turn on power. It boots pretty much immediately into a GUI, a QT one, not ncurses, which means there’s basically a full OS running underneath. The choices of OSes available for downloading are quite diverse. Raspbian (the largest at 1003MB), BerryWebServer (the smallest at 5MB), the One Laptop per Child OS, also NOC-PS, for setting up a server farm (up to 25 servers / virtual hosts). It’s a well put together piece of software, the Berryboot community should give itself a big pat on the back. This is what the uEnv.txt looks like:-
This is cmdline.txt
which is totally different from what we’ve seen before. Notice the *conditional* in the first line. I wonder what else can be used. Interestingly, config.txt, which is normally a long list of options, is small enough to post here in its entirety
There is a tgz file called shared, but I didn’t see anything to do with booting there. It looks like files that are common to all OSes, and probably explains partly why the rest is so small.
So I loaded 6 OSes, which was the maximum I could put on the 2GB USB stick I had.
I’m definitely going to be playing with this a lot more. The biggest let down was my own setup. I don’t have a monitor with a HDMI interface, so I have to put the composite output to a decoder which turns it into SVGA, but it comes out with huge text and windows that are bigger than the screen sometimes, which gets very interesting when you cannot close or minify a window.
One thing to note. The link you provided is perfect for me, because I’m putting it on a Pi B, but they have a newer version for the Pi 2, which probably uses the four cores
Pingback: Infrastructure R.Pi Build Progress | Musings from the Chiefio