Rebuilding GIStemp Engine and Model Platform

OK, I’ve been low on postings for the last week or so (despite having the weekend to work in and despite having about 50 tabs “open and waiting” to be written up. Why? Well, that’s the subject of this posting… Sometimes you just must take time to work on the tools.

I have an “interesting collection” of hardware. When I first got back to The Lab here, the old Compaq Evo had “had issues” ranging from Mephis scrogging the boot block for the NT partition to a flat out failure (eventually traced to a loose memory stick). I’d gotten all that working (details down that rabbit hole here: https://chiefio.wordpress.com/2015/06/09/wheezy-debian-seems-stable-on-evo-now/ in the ‘tale of woe’ link).

And Debian has been, mostly, stable since the last iteration. I’ve installed a half dozen other windows managers and desktops on it, and a couple seem to be completely stable (so far); though I have still had the odd hang return in my default XFCE choice. All in all, workable as a play space / browser home, but not quite the “industrial strength” stability I prefer for “real work”.

Along the way, I’d bought a $70? dollar XP box at Weird Stuff Warehouse (one of my favorite haunts for old and good-but-cheap hardware). Despite them having a few dozen boxes with Linux already installed, I chose an interesting one that was both 64 bit and XP. Why? Because at the time the EVO was DOA and I thought I might need to just restore the data to a different XP box (as that was the platform / application set it was all working with…). After recovering the EVO, that need diminished (though I still want it as a ‘spare’). My secondary goal had been to make it the “New GIStemp box” with decent FORTRAN compiler and all, running LINUX.

That was where “Tale Of Woe Two” kicked in.

It is an Antek box with an ASUS mother board. I read the label on it (each box at Weird Stuff is labeled with tech info like disk size, memory size, motherboard, OS / Level, and more). I knew what I was buying, or so I thought. AMD Sempron processor. More than enough memory. SATA disk. etc.

What I didn’t know was “That thing that bites me in Linux Land” so often. Of all the motherboards in all the world, this ASUS board had to walk into my bar… It is an ASUS M2V-M with an odd video chipset on it. Turns out most flavors of Linux don’t know about it on their Live-CDs or Live-DVDs. So “no joy” on the “boot and go” for most of them.

That also means no go on the boot then install. It’s back to long-hand land and fixing up the video drivers supported by that release. A somewhat painful old school process I’d rather not do again. So I didn’t. After a cruise through my canonical collection of live-disks (Crunch Bang, Knoppix, SliTaz, Debian, Puppy-several variations, Gentoo, Arch, etc. etc.) there were exactly 2 that came up fine, native, with the right video driver. CentOS and SystemRescueCD.

I’ll spare you the long and painful series of attempts to get something to run from a USB Stick (way too slow and prone to boot one day, not the next, along with CentOS having heavy write activity and will likely burn the USB stick fast. Limited write cycles on USB, after all). As near as I can tell, getting this particular mother board / BIOS / USB set to load the right stuff to get a boot to happen (and being 64 bit in a world of old 32 bit pieces) to actually reliably boot something on USB is “not easy”.

I’d been trying to avoid an install to the hard disk as that is prone to sometimes wiping your old Windoz partitions and / or breaking the boot sector / MBR. (Master Boot Record). I suspect some of the USB fail-to-boot were over things like MBR vs syslinux vs 32 bit vs 64 bit bios/CPU… And I didn’t want that to come up on this hard disk / XP backups copy. But…

For two days I’d worked on getting a very nice CentOS installed to an older Western Digital 111 GB USB disk. Works fine on the EVO. The same 32 bit CentOS Live-CD also boots on the Asus/Antek, so I’d expected it to work there, too. No joy. Could not get it to boot. Not with BIOS settings to boot from USB, nor with Plop Bootmanager. Plop works fine for other things on the Antek/Asus, and works with the WD111 disk on the Evo 32 bit, but not with it on the Antek/Asus.

That was last night and it is a stellar example of my mantra:

“Why? Don’t ask why. Down that path lies insanity and ruin… -E.M.Smith”

Sure, I could spend a few days at it and probably work it out, but I decided otherwise.

CentOS Install Style

CentOS is oriented toward Enterprise scale operations. Big companies. Professional staffs. R&D Labs. It is prone to being fairly stable, highly reliable, and a bit out of date. Little “bleeding edge” here, but lots of “just works”, along with a “tasteful” professional look and feel. Nice. However…

The install process is a bit, um, “authoritarian”. It just KNOWS what you really ought to be doing and it is not about to coddle you with a bunch of choices and silly wrong options and warnings. After all, you ought to have done this 100 times already during your internship…

So the root file system WILL BE EXT4. No debate. No options. Fine on large servers. Chews writes to a USB stick like crazy. ( I had one working from a USB stick on the ASUS, for a while, and it was just bog slow. Painfully so.) EXT3 and EXT4 are journaling file systems. That means they make an entry to disk “I’m going to write file FOO”, then they write it, then they write “I have succeeded and finished writing file FOO”. This makes recovery from surprise shutdowns easier, but at the expense of 3 writes for one block written. USB / SD Cards / FLASH likes to write a large (very large…) block of data all at once, so it will ‘read out the nearby stuff, update the bits in the middle, then write the giant block back’ for a small spot of update. Now multiply that by 3. Just a killer. So while I’d wanted to make root EXT2 to cut those writes to 1/3, that was not allowed.

Similarly, it gives you some disk formatting choices as a bullet list. Blow it all away. Blow away any prior Linux partitions but not the Windows (FAT / NTFS) partitions. Slide some other partition out of the way. A couple of others, and “custom”. BUT custom will not let you make a SWAP partition on a USB drive. Even a real hard disk USB drive. It makes a lot of sense not to swap to flash as the writes will kill it, but not so a real disk on USB. (About a decade and 1/2 ago was my first USB Linux. Swap killed the stick in about a week…) But in a large enterprise they just KNOW swap will be on your large hard disk…

I did find that by using the SystemRescueCD I could build the partitions I wanted (large “/” or root, nice 4 GB swap, and large /home; and it let me shrink the XP partition to about 1/2 the disk without problems). Then, I thought, I can just use the “install to existing Linux partitions” in CentOS and move on.

I was almost right.

The one I chose looked like it meant “I’ll use what you gave me”, but in true “I know better authoritarian” style, it knew better. The good news is that it knew enough to keep hands off the XP partition. (For some reason the ‘move it over’ option didn’t love me, so I’d had to resort to the SystemRescueCD to slide XP over… IIRC it was because the XP partition was not seen as “empty”. CentOS install didn’t look inside the partition and resize it, just wanted to slide things over full sized.) At any rate, I’d gotten past that, had my partitions ready, and said “use them”.

And it did. It sucked up all that disk space, turned it into sizes it liked instead (after all, it knows best…) nuked swap (since it doesn’t think old fashioned real swap partitions are the way to go) and spit out a file layout using Volume Groups…

Now don’t get me wrong. I like Volume Groups; especially in large shops where downing volumes and swapping what lives on which disks are common problems. You can do RAID like things with them and dynamically resize partitions. Lots of good stuff. But for a 1 disk 2 partitions plus swap it is just way overkill. And I would like to have been asked before being reformatted…

Now you can get that layout as you like it (and I’d done so before) using the “custom layout” option. So that looks like the ONLY way to stop it from doing what it wants to your disk or open space. Sigh. It was either “start over with partitioning” or just move on and live with Volume Groups.

I moved on.

Besides, I rationalized, I could use the review / practice with that part of SysAdmin. And on my “someday list” is set up a file server with RAID and networked disks and a cluster and… VG is well suited to that kind of use. Figure I’ll get to it in about 3 more years. Maybe…

Just be advised that CentOS has firm ideas about what to do and often does NOT ask your opinion.

Moving On

That was last night about 2 or 3 AM. This morning at about 9 AM I woke up and “moved in” to the machine. I’ve gotten the WD USB disk to mount onto it, so all the data from archives I’d loaded into that CentOS install are visible. I’ve moved some of the data archives onto hard disk on the ASUS (as for some reason the AUS is surprisingly slow with USB disks (even in Windoze) and keeps nagging me that I can speed them up by plugging into one of the fast ports, that it lists, but any port I try gets the same message… but it’s fast enough for moving stuff and light duty).

Then, about an hour ago, I decided to start building the development environment.

On the old Red Hat 7.2 system that was the GIStemp station, that was a major pain. On CentOS, it was a cake walk. I’d planned to just keep using the Vectra that replaced that old White Box (on the same Red Hat 7.2 release) but that motherboard had the CD/DVD drive fail at some point, so it was deprecated too. Thus the move to something newer.

Instead of finding and downloading Python and FORTRAN language suites / compilers and building them all by hand, it was just one command. Here’s a link to a page saying what to do, the command, and a screen capture or two of the build happening:

The key is to realize you are getting all of the “core development tools” and not just FORTRAN.

http://www.cyberciti.biz/faq/centos-linux-install-gcc-c-c-compiler/

You need to install ‘Development Tools’ group on RHEL/CentOS/Fedora/Scientific/Red Hat Enterprise Linux. These tools include core development tools such as automake, gcc, perl, python, and debuggers which is required to compile software and build new rpms:
[…]
Installation

Open the terminal or login over ssh session and type the following command as root user:

# yum groupinstall 'Development Tools'

That’s it. Here’s the screen shots:

Screenshot of CentOS install during gfortran portion

Click for a very large easy to read version, but realize there are a lot of windows in that screenshot… The foreground middle right on line (23/82) shows gcc-fortran in the mix. In the left middle part of the panel is that page with the one line command on it. Up top are my tabs and menu bars for CentOS and FireFox (where you can see the first pages I bookmarked ;-) and at the very bottom you can see a bit of a system performance monitor and a terminal window where I was moving a copy of GHCNv3 from USB disk to hard disk.

Now you know my usual working style. Half dozen panels open, with a system status monitor, a “top”, a working terminal, a root terminal, a browser, a background major task, and some foreground task.

Here’s a similar screen cap where it has gotten to the install Perl stage:

Development Tools Install at Perl step

This mostly shows that it is installing a lot more than just FORTRAN.

In the upper right is the Perl and related tools being installed. Just below it is the upper edge of a terminal window running “top” that shows 46.5% CPU for “user”, and that I’ve used most of the real memory (though much of those blocks now released) and have a 6 GB Swap area with only 84 Meg used. Why 6 GB for a 1 GB memory machine? CentOS decided I only needed 2 GB, and put that on the VG volume anyway. I know I like to have a LOT of tabs open in a browser, and they each take memory, so I’d had a 4 GB real swap partition that it blew away… So I added a file in /usr/SWAP of 4 GB and did a mkswap / swapon /usr/SWAP to get the added space (and show it who was really boss ;-)

I foregrounded the full performance monitor that comes with it. Nice. It shows only 635 MB of memory used while “top” shows 956 MB. They are both right. It has to do with how stale blocks and cache is reported, IIRC.

Finally, at the end of the install, I brought the system config panel to the front to document what the hardware might be.

Dev Tools Install finishes on CentOS

And with that, I’m starting a new set of work on GHCN and GIStemp. Don’t know exactly what I’ll start with. There are many directions to go. V1 vs V2 vs V3? Make my own unadjusted composite V0? Compare new GIStemp to old? Who knows.

There will be about a week more of unpacking old copies of the data. Downloading a new pristine copy. Unpacking the old GIStemp sources and build. Cleaning up some old junk (and maybe figuring out again how it worked when I wrote some trick analysis bit of FORTRAN :-) and generally settling in.

I’m also really enjoying having a Real Linux Workstation again as my Daily Driver. The tablet is OK as a ‘read mostly; comment terse’ portable thing at the coffee shop. The Chromebox is a very reliable does-just-enough to do postings straight jacket with guards watching. The XP Evo is, well, XP. And the Debian is comfortable, but I’m a bit wary of things that sporadically hang for no reason; even if mostly fixed now. This box, despite the pain and suffering to reach this point, is acting rock solid.

Not a glitch. Not a hang. Not a slowdown or hiccup that isn’t clearly from my loading it to the hilt with parallel tasks to do.

So, with that, I’m going to shut it down and take a dinner break. (Well, really, a “cook dinner for the spouse who wants The Butler to stop playing with his computer” break…) and come back a bit later to continue unpacking data and code, and maybe in the wee hours of the cool night do my first trial GIStemp compile.

Subscribe to feed

Advertisement