Raspberry Pi GIStemp – It Compiled

I’ve not done any testing yet, just a first cut “unpack and compile”.

The good news is that it did, in fact, compile. And STEP0 ran.

I had the same two “Makefile” issues from the port to Centos (Red Hat for the Enterprise). No surprise as it was an unpack of the same tar file archive with the same two issues in it. A bogus “:q” from a botched exit from the editor in one source file, and the need to remove a trace directive from the compile statement on another. After that, it compiled fine with one warning.

That warming might be a significant bug in the GIStemp code. I’ll need to investigate it further. I don’t remember if any of the other compilations found that type mis-match. The importance of it is still TBD and might be nothing.

Here’s the captures on what I did:

Install FORTRAN

Whereas in the Centos install, FORTRAN is part of “build-essentials”, it was not included in the R.Pi version. That required a specific call out of gfortan as a package. Here’s the call to “apt-get install gfortran” and the result.

root@RaPiM2:/WD/home/BUPSfromGTemp# apt-get install gfortran

Reading package lists… Done
Building dependency tree
Reading state information… Done
The following extra packages will be installed:
gfortran-4.6
Suggested packages:
gfortran-doc gfortran-4.6-doc libgfortran3-dbg
The following NEW packages will be installed:
gfortran gfortran-4.6
0 upgraded, 2 newly installed, 0 to remove and 0 not upgraded.
Need to get 4,834 kB of archives.
After this operation, 12.0 MB of additional disk space will be used.
Do you want to continue [Y/n]? y
Get:1 http://mirrordirector.raspbian.org/raspbian/ wheezy/main gfortran armhf 4:4.6.3-8 [1,134 B]
Get:2 http://mirrordirector.raspbian.org/raspbian/ wheezy/main gfortran-4.6 armhf 4.6.3-14+rpi1 [4,833 kB]
Fetched 4,834 kB in 34s (140 kB/s)
Selecting previously unselected package gfortran-4.6.
(Reading database … 89899 files and directories currently installed.)
Unpacking gfortran-4.6 (from …/gfortran-4.6_4.6.3-14+rpi1_armhf.deb) …
Selecting previously unselected package gfortran.
Unpacking gfortran (from …/gfortran_4%3a4.6.3-8_armhf.deb) …
Processing triggers for man-db …
Setting up gfortran-4.6 (4.6.3-14+rpi1) …
Setting up gfortran (4:4.6.3-8) …
update-alternatives: using /usr/bin/gfortran to provide /usr/bin/f95 (f95) in auto mode

root@RaPiM2:/WD/home/BUPSfromGTemp# which gfortran
/usr/bin/gfortran

So I’ll need to add that “apt-get install gfortran” to my Raspberry Pi software build script.

Here’s the change to the top of the Makefile to tell it to use gfortran instead of the FORTRAN compilers of old:

FC=/usr/bin/gfortran
FC95=/usr/bin/gfortran
FC77=/usr/bin/gfortran

#FC=/usr/local/bin/g95
#FC95=/usr/local/bin/g95
#FC77=/usr/bin/f77
SRCDIR=.
BINDIR=../bin

I inserted “script variables” to use gfortran for all three of “generic FORTRAN”, “FORTRAN 95”, and “FORTRAN 77” compilers. Commenting out the older assignments that used g95 for the first two, and a real f77 for the third. (gfortran being smart enough to do F77 things with F77 code). But at least, due to having made that Makefile, I didn’t have to edit every single script that their version used where each program was complied anew each time it was run, so EVERY run script had a call out to the compiler to use… Really crappy “technique”…

Now with gfortran installed and located as in /usr/bin/gfortran, I went to the GIStemp src directory (remember that I moved sources to one directory and made a “Makefile” as opposed to leaving them scattered all over the place in run-scripts as built). At that point, the two edits were done and I typed “make”.

Here’s the errors that showed the two things I had to fix. Yes, the compiler nicely tells you what you screwed up:

/usr/bin/gfortran -o ../bin/toANNanom.exe ./toANNanom.f
./toANNanom.f:40.1:

:q
1
Error: Non-numeric character in statement label at (1)
./toANNanom.f:40.1:

:q
1

In the program “toANNanom.f” when last I’d edited it back in about 2010, I’d tried to leave the “vi” (visual editor) and not realized I was in “insert” mode. That “:q” is the vi command to “exit”. So I’d exited from edit mode, and then saved and exited the program, but left that artifact inserted. OK. Took it out. (And yes, one then needs to go regression test the code against earlier copies to make sure nothing important got messed up).

The next run found the trace directive in the Makefile and didn’t like it.

pi@RaPiM2 /WD/home/BUPSfromGTemp/gistemp/src $ make
/usr/bin/gfortran -o ../bin/sorts.exe ./sorts.f
/usr/bin/gfortran -o ../bin/antarc_comb.exe ./antarc_comb.f
/usr/bin/gfortran -o ../bin/cmb2.ushcn.v2.exe ./cmb2.ushcn.v2.f
/usr/bin/gfortran -o ../bin/cmb.hohenp.v2.exe ./cmb.hohenp.v2.f
/usr/bin/gfortran -o ../bin/dump_old.exe ./dump_old.f
/usr/bin/gfortran -o ../bin/hohp_to_v2.exe ./hohp_to_v2.f
/usr/bin/gfortran -o ../bin/USHCN2v2.exe ./USHCN2v2.f
#echo “USHCN2v2.f is compiler senstive at run time. 1 digit rounding variation
/usr/bin/gfortran -o ../bin/dif.ushcn.ghcn.exe ./dif.ushcn.ghcn.f
#echo “Still a custom makefile inside STEP1”
/usr/bin/gfortran -o ../bin/flags.exe ./flags.f
/usr/bin/gfortran -o ../bin/padjust.exe ./padjust.f
/usr/bin/gfortran -o ../bin/toANNanom.exe ./toANNanom.f
/usr/bin/gfortran -o ../bin/PApars.exe ./PApars.f ./tr2.f ./t2fit.f
/usr/bin/gfortran -o ../bin/split_binary.exe ./split_binary.f
/usr/bin/gfortran -o ../bin/text_to_binary.exe ./text_to_binary.f
/usr/bin/gfortran -o ../bin/trim_binary.exe ./trim_binary.f
/usr/bin/gfortran -o ../bin/invnt.exe ./invnt.f
/usr/bin/gfortran -o ../bin/trimSBBX.exe ./trimSBBX.f
/usr/bin/gfortran -o ../bin/toSBBXgrid.exe ./toSBBXgrid.f
/usr/bin/gfortran -o ../bin/zonav.exe ./zonav.f
/usr/bin/gfortran -o ../bin/annzon.exe ./annzon.f
/usr/bin/gfortran -o ../bin/convert1.HadR2_mod4.exe ./convert1.HadR2_mod4.f
/usr/bin/gfortran -o ../bin/convert.HadR2_mod4.upto15full_yrs.exe ./convert.HadR2_mod4.upto15full_yrs.f
/usr/bin/gfortran -ftrace=full -o ../bin/SBBXotoBX.exe ./SBBXotoBX.f
f951: error: unrecognized command line option ‘-ftrace=full’
Makefile:91: recipe for target ‘SBBXotoBX’ failed
make: *** [SBBXotoBX] Error 1
pi@RaPiM2 /WD/home/BUPSfromGTemp/gistemp/src $ vi Makefile

I have a couple of notes to myself about STEP1 having it’s own makefile that I need to go set up and test ( it uses Python for some special things and needs some more complicated set up) and that USHCN2V2.f has a rounding error sensitivity to what compiler is used (so needs specific testing / validation).

Then, down near the bottom, you can see that it complains about the -ftrace=full directive. OK, some compiler flags are different for doing traces. For now I just pulled that trace out. I don’t remember why I wanted to trace it 5 years ago anyway ;-) Then the compile ran to completion:

pi@RaPiM2 /WD/home/BUPSfromGTemp/gistemp/src $ make
/usr/bin/gfortran -o ../bin/sorts.exe ./sorts.f
/usr/bin/gfortran -o ../bin/antarc_comb.exe ./antarc_comb.f
/usr/bin/gfortran -o ../bin/cmb2.ushcn.v2.exe ./cmb2.ushcn.v2.f
/usr/bin/gfortran -o ../bin/cmb.hohenp.v2.exe ./cmb.hohenp.v2.f
/usr/bin/gfortran -o ../bin/dump_old.exe ./dump_old.f
/usr/bin/gfortran -o ../bin/hohp_to_v2.exe ./hohp_to_v2.f
/usr/bin/gfortran -o ../bin/USHCN2v2.exe ./USHCN2v2.f
#echo “USHCN2v2.f is compiler senstive at run time. 1 digit rounding variation
/usr/bin/gfortran -o ../bin/dif.ushcn.ghcn.exe ./dif.ushcn.ghcn.f
#echo “Still a custom makefile inside STEP1”
/usr/bin/gfortran -o ../bin/flags.exe ./flags.f
/usr/bin/gfortran -o ../bin/padjust.exe ./padjust.f
/usr/bin/gfortran -o ../bin/toANNanom.exe ./toANNanom.f
/usr/bin/gfortran -o ../bin/PApars.exe ./PApars.f ./tr2.f ./t2fit.f
/usr/bin/gfortran -o ../bin/split_binary.exe ./split_binary.f
/usr/bin/gfortran -o ../bin/text_to_binary.exe ./text_to_binary.f
/usr/bin/gfortran -o ../bin/trim_binary.exe ./trim_binary.f
/usr/bin/gfortran -o ../bin/invnt.exe ./invnt.f
/usr/bin/gfortran -o ../bin/trimSBBX.exe ./trimSBBX.f
/usr/bin/gfortran -o ../bin/toSBBXgrid.exe ./toSBBXgrid.f
/usr/bin/gfortran -o ../bin/zonav.exe ./zonav.f
/usr/bin/gfortran -o ../bin/annzon.exe ./annzon.f
/usr/bin/gfortran -o ../bin/convert1.HadR2_mod4.exe ./convert1.HadR2_mod4.f
/usr/bin/gfortran -o ../bin/convert.HadR2_mod4.upto15full_yrs.exe ./convert.HadR2_mod4.upto15full_yrs.f
/usr/bin/gfortran -o ../bin/SBBXotoBX.exe ./SBBXotoBX.f
./SBBXotoBX.f:181.39:

CALL SREAD(10,MNOW,AVG(I1TIN,NC),XTRL,MNEW)
1
Warning: Type mismatch in argument ‘itrl’ at (1); passed REAL(4) to INTEGER(4)

Here’s the relevant part of the Makefile:

all: inputsort step0 step1 step2 step3 step4_5

step0 : antarc cmb2 cmbho dumpold hotov2 u2v2 dif.ushcn.ghcn

step1:
#echo “Still a custom makefile inside STEP1”

step2: flags padjust toANNanom PApars split_binary text_to_binary trim_binary invnt

step3: trimSBBX toSBBXgrid zonav annzon

step4_5: convert1.HadR2_mod4 convert.HadR2_mod4.upto15full_yrs SBBXotoBX

“Make”ing all does the input sort, then steps in order ending with step4_5. Further down, we see that step4_5 includes three parts, the last of which was SBBXotoBX that complained about a type mis-match with a warning. So it got through the compiles OK.

Since sometimes it is OK to pack a real into an int, and sometimes it is an error, I’ll need to get very familiar with that code to see which it is. As that’s part of the very last grid box portion, I’m not likely to bother.

This is the 5 year old version of GIStemp. I mostly wanted to compile it just to see if it would, generally, go. I’m most likely to get the newer version (that uses different input data now too) and port it first before sinking too much more time into this old version.

At some much later time, I might want to compare the two. That would be a good time to come back to this potential error in their old code.

Trial Run of STEP0

I also did a trial run of STEP0. Turns out that it doesn’t like [[ in shell scripts even if #!/bin/sh is called out. I changed the shell scripts to have “#!/bin/bash” and everything then worked as expected. Oh, and I also had to change “tail +100” to “tail -n 100” in the script as the options are different (and I ought to check that those are really the same between the two versions… it’s been a while since I used each version of “tail” – that gives the last set of lines in a file.)

Most of the time, it “pegs” one core on the FORTRAN. Here’s a grab of “top” showing USHCN2v2.exe at 99.8%

Top - 06:56:37 up  9:26,  4 users,  load average: 1.30, 0.92, 0.81
Tasks: 148 total,   2 running, 146 sleeping,   0 stopped,   0 zombie
%Cpu(s): 25.7 us,  0.6 sy,  0.0 ni, 73.8 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
KiB Mem:    949408 total,   928900 used,    20508 free,    16272 buffers
KiB Swap:  3174392 total,       12 used,  3174380 free,   481096 cached

  PID USER      PR  NI  VIRT  RES  SHR S  %CPU %MEM    TIME+  COMMAND            
16388 pi        20   0  2964  532  464 R  99.8  0.1   0:17.22 USHCN2v2.exe       
14405 pi        20   0  588m 289m  41m S   4.6 31.3  55:55.12 iceape-bin         
12141 root      20   0 66284  39m 9132 S   0.7  4.3   8:09.38 Xorg               
15272 pi        20   0  4676 2460 2076 R   0.7  0.3   0:16.34 top                
12243 pi        20   0  106m  16m  13m S   0.3  1.8   0:48.41 lxterminal         
14518 mysql     20   0  308m  34m 7844 S   0.3  3.8   0:08.20 mysqld             

Most of the memory used is for stale buffers. The performance tool showed about 375 MB actually used by everything at this point.

And here is the result of the run on STEP0:

pi@RaPiM2 /WD/home/BUPSfromGTemp/gistemp/STEP0 $ time ./do_comb_step0.sh v2.mean
Clear work_files directory? (Y/N) y
Bringing Antarctic tables closer to input_files/v2.mean format
collecting surface station data
… and autom. weather stn data
… and australian data
replacing ‘-‘ by -999.9, blanks are left alone at this stage
adding extra Antarctica station data to input_files/v2.mean
created v2.meanx from v2_antarct.dat and input_files/v2.mean
removing pre-1880 data:

GHCN data:

removing data before year 1880.00000
created v2.meany from v2.meanx

replacing USHCN station data in v2.mean by USHCN_noFIL data (Tobs+maxmin adj+SHAPadj+noFIL)
reformat USHCN to v2.mean format
extracting FILIN data

-rw-r–r– 1 pi pi 20924805 Jul 29 06:56 hcn_doe_mean_data_fil
-rw-r–r– 1 pi pi 81574970 Feb 17 2010 input_files/hcn_doe_mean_data

getting inventory data for v2-IDs

-rw-r–r– 1 pi pi 32967 Sep 16 1998 input_files/ushcn.tbl

After the sort of ushcn.tbl into ID_US_G

Doing ../bin/USHCN2v2.exe the ***OLD*** USHCN.v2 Version

USHCN data end in 2007
finding offset caused by adjustments
extracting US data from GHCN set
removing data before year 1980.00000
getting USHCN data:
-rw-r–r– 1 pi pi 11111793 Jul 29 06:56 USHCN.v2.mean_noFIL
-rw-r–r– 1 pi pi 10189487 Jul 29 06:57 xxx
doing dump_old.exe
removing data before year 1880.00000
-rw-r–r– 1 pi pi 10124653 Jul 29 06:57 yyy
Sorting into USHCN.v2.mean_noFIL
-rw-r–r– 1 pi pi 10124653 Jul 29 06:57 USHCN.v2.mean_noFIL
done with ushcn

created ushcn-ghcn_offset_noFIL

Doing cmb2.ushcn.v2.exe

created v2.meanz

replacing Hohenspeissenberg data in v2.mean by more complete data (priv.comm.)
disregard pre-1880 data:

At Cleanup

created v2.mean_comb

-rw-r–r– 1 pi pi 44774884 Jul 29 06:59 to_next_step/v2.mean_comb

move this file from to_next_step/. to ../STEP1/to_next_step/.
Copy the file to_next_step/v2.mean_comb to ../STEP1/to_next_step/v2.mean_comb? (Y/N) y

and execute in the STEP1 directory the command:
do_comb_step1.sh v2.mean_comb

real 4m48.070s
user 4m5.370s
sys 0m10.660s
pi@RaPiM2 /WD/home/BUPSfromGTemp/gistemp/STEP0 $

At the very bottom is the result of the “time” command that tells you elapsed time and how much CPU time was used. So this step took 4 minutes 48 seconds to run, used 4 minutes 5 seconds of user CPU time and 10 seconds of system CPU time. Essentially pegging one core for 5 minutes. Not bad. IIRC, each step is about the same run time. So this would be in keeping with the 20 to 30 minutes end-to-end I think I remember from before, on the old x86 CPU box with faster disks.

This was on a USB disk with EXT3 partitions and I added a swap partition which you can see in the “top” listing as well. The R.Pi uses a 1 GB swap file, but swapping to SD card is just soo wrong, so I added a real partition… that never gets used… oh well…).

Just in case anyone else wants to know how to do that, here’s the /etc/fstab lines:

/dev/sda3       swap            swap    sw,pri=1024       0       0
/dev/sda4       /WD/home        ext3    defaults          0       1

That “pri=1024” is the priority for that swap partition. It ranges from zero (last) to 30,000 ish (first). I just set it to 1k so it comes ahead of the swap file set by default. That ought to direct any swapping first to my real USB disk, only when that 2 GB get used up, spill over to their 1 GB file. But if I’m ever using 1 GB of memory and 2+GB of swap, I’ll have other issues to worry about ;-)

Also, since GIStemp repeatedly writes 10 to 40 MB files, I decided to not burn that on my expensive fast 64 Gig SD card and instead put it on a real USB disk that doesn’t care about write cycles.

In Conclusion

That’s basically it. Initial unpack and install done. Compiler check and install done. Make done, and make file updated / bugs ironed out. STEP0 first run completed. Next step to do the STEP1 “make” that is “special” and try doing a couple of runs end to end. Then download a fresh copy of the source, and see how much has changed in 1/2 decade.

Don’t know when I’ll get all that done. At some point I’m supposed to go looking for a real job to pay the rent. And the garden calls my name when the weather is nice. Then there are the 1001 other interesting things happening to / in the world. But I’m not going to just let GIStemp lay there. I’ll get to it. Maybe mañana ;-)

I do find it rather fun that GIStemp compiles and runs at a reasonably fast clip on a $60 computer. Kind of makes you wonder what they need all the super duper new expensive computers for…

Subscribe to feed

About E.M.Smith

A technical managerial sort interested in things from Stonehenge to computer science. My present "hot buttons' are the mythology of Climate Change and ancient metrology; but things change...
This entry was posted in AGW GIStemp Specific, GISStemp Technical and Source Code, Tech Bits and tagged , , , . Bookmark the permalink.

11 Responses to Raspberry Pi GIStemp – It Compiled

  1. E.M.Smith says:

    I sat down at the CentOS computer and was going to do something else… then it occurred to me that I could use GIStemp as a kind of benchmark if I grabbed timing for STEP0 from it, too. So I did. It completed in about 1/4 the time and looking at “top” showed sometimes the CPU was pegged at near 100%, but sometimes it was down at 50% ish range. Oddly, this implies that the R.PiM2 is about the same speed as this 64 bit AMD CPU, but spread over 4 cores so when it pegs just one the throughput rate is about 1/4. Interesting…

    I’ll do some other comparisons later. Things like byte size of binaries and what have you. But for now, the basic conclusion is that for about the same money I got two boxes of about the same performance. One, the new R.PiM2 and the other the old ASUS /Antek box. Both have about the same memory and “disk” space too. Hmmmm….. But the ASUS board with the AMD chip in it runs single threaded tasks 4 x faster as it is ‘core limited’ to just one of 4 cores on the R.PiM2. Interesting…

    Here’s the run from the CentOS box:

    [chiefio@CentosBox STEP0]$ time ./do_comb_step0.sh v2.mean
    Clear work_files directory? (Y/N) y
    Bringing Antarctic tables closer to input_files/v2.mean format
    collecting surface station data
    … and autom. weather stn data
    … and australian data
    replacing ‘-‘ by -999.9, blanks are left alone at this stage
    adding extra Antarctica station data to input_files/v2.mean
    created v2.meanx from v2_antarct.dat and input_files/v2.mean
    removing pre-1880 data:

    GHCN data:

    removing data before year 1880.0000
    created v2.meany from v2.meanx

    replacing USHCN station data in v2.mean by USHCN_noFIL data (Tobs+maxmin adj+SHAPadj+noFIL)
    reformat USHCN to v2.mean format
    extracting FILIN data

    -rw-rw-r–. 1 chiefio chiefio 20924805 Jul 29 11:30 hcn_doe_mean_data_fil
    -rw-rw-r–. 1 chiefio chiefio 81574970 Feb 17 2010 input_files/hcn_doe_mean_data

    getting inventory data for v2-IDs

    -rw-r–r–. 1 chiefio chiefio 32967 Sep 16 1998 input_files/ushcn.tbl

    After the sort of ushcn.tbl into ID_US_G

    Doing ../bin/USHCN2v2.exe the ***OLD*** USHCN.v2 Version

    USHCN data end in 2007
    finding offset caused by adjustments
    extracting US data from GHCN set
    removing data before year 1980.0000
    getting USHCN data:
    -rw-rw-r–. 1 chiefio chiefio 11111793 Jul 29 11:30 USHCN.v2.mean_noFIL
    -rw-rw-r–. 1 chiefio chiefio 10189487 Jul 29 11:30 xxx
    doing dump_old.exe
    removing data before year 1880.0000
    -rw-rw-r–. 1 chiefio chiefio 10124653 Jul 29 11:30 yyy
    Sorting into USHCN.v2.mean_noFIL
    -rw-rw-r–. 1 chiefio chiefio 10124653 Jul 29 11:30 USHCN.v2.mean_noFIL
    done with ushcn

    created ushcn-ghcn_offset_noFIL

    Doing cmb2.ushcn.v2.exe

    created v2.meanz

    replacing Hohenspeissenberg data in v2.mean by more complete data (priv.comm.)
    disregard pre-1880 data:

    At Cleanup

    created v2.mean_comb

    -rw-rw-r–. 1 chiefio chiefio 44776809 Jul 29 11:31 to_next_step/v2.mean_comb

    move this file from to_next_step/. to ../STEP1/to_next_step/.
    Copy the file to_next_step/v2.mean_comb to ../STEP1/to_next_step/v2.mean_comb? (Y/N) y

    and execute in the STEP1 directory the command:
    do_comb_step1.sh v2.mean_comb

    real 1m8.634s
    user 0m48.199s
    sys 0m2.925s
    [chiefio@CentosBox STEP0]$

    Here we see that it was 1 min 9 seconds elapsed, but only 48 seconds user CPU, so ‘not pegged’ for 21 of the seconds in that elapsed time.

    I’m not particularly excited about “saving” 3.x minutes on a run of a stage. Either one of the two is “fast enough”, but if I’m looking for fast turns, the One Big CPU does better. (No idea how it would work on a multicore multithreaded new Intel WizBang CPU, but likely would still be single threaded for most things… FORTRAN tends to that way unless coded for parallel execution).

    It is also clear that this AMD 64 bit CPU is a LOT faster than the old AMD 400 MHz x86 chip in the old GIStemp station. In that 1 minute, it moves around a LOT of data. I do “ls” listings on some of the intermediate files as they are created in the run log above. For example:

    created v2.mean_comb

    -rw-rw-r–. 1 chiefio chiefio 44776809 Jul 29 11:31 to_next_step/v2.mean_comb

    Tells you the prior step made a file called “v2.mean_comb” (or the combined form of the version 2 ‘mean’ temperature file). Then the “ls -l” or listing with long form data of it shows it is read write by me, has 44,776,809 bytes in it (or about 44 MB) and was made today July 29 at 11:31 A.M. It is in the directory “to_next_step” and is named “v2.mean_comb”.

    Slugging around several 44 MB copies and a few 10 MB copies in about a minute ain’t bad. Doing it in about 4 to 5 minutes on a R.Pi board with a USB disk is not a whole lot worse. I’m happy with either of them as an experimental platform.

    Well, there’s your data… do with it what you will. ;-)

  2. p.g.sharrow says:

    Not bad for a cheap kids toy! Lol! pg

  3. Don’t waste time on that itrl[1] real/int warning. It’s not a bug, just the sort of thing old school FORTRAN programmers did, and totally pointless by the 80’s, whatever justification it may have had in the 60’s. itrl has both integer and real values, but these are handled consistently in such a way that no type conversion takes place inadvertently. I remember checking this carefully when rewriting the Gistemp FORTRAN code in C#. It seemed totally unnecessary obfuscation to “save” a handful of bytes. But that’s FORTRAN style for you. I started with FORTRAN II in 1965, and reformed in the early 1990’s. I’m sorry to say that some of the Gistemp FORTRAN code seemed only too familiar in style.

  4. E.M.Smith says:

    @Peter O’Neill:

    Thanks! You just saved me a few dozen hours of my life!!

    I also remembered that I added the “trace” directive to the prior makefile in preparation for tracing just what happened in that program, but couldn’t work up the ambition to “go there”… likely saving myself yet more time ;-)

    My major intent had just been to see if this stuff would compile and run at all; figuring that most of it will likely have been carried forward “as is” into the present version of GIStemp. With the “it complies” and “it runs” out of the way and the “is it a bug or a feature?” answered; I’m likely to just head into “get the new version and port it” phase.

    Though I just might copy the whole directory onto the SD card and do one run there, just for timing and performance values. I have this fantasy of porting GIStemp (and maybe some of the climate models) onto the Raspberry Pi and offering SD cards to folks “ready to go” for those who want to play with it a little but not wanting to get into the porting software game… It’s likely way too thin a “market” to be a business… but might save somebody some time.

    Besides, the idea of “GIStemp on a chip” running on a $60 computer has a certain “damning with faint praise” aspect I find attractive ;-)

    Though I do think it would burn the “chip writes lifetime” rather fast…

    Per FORTRAN:

    I started with FORTRAN IV in the early ’70s and actually like it. Done right it’s a good language for fixed format data moving and math problems. Unfortunately, it is often not done right. As my first ever programming class, I was blessed with exceptional instruction. It was offered as an Engineering division class, taught by Engineers for Engineers. The class was in many ways structured around “How can you shoot your whole foot off with this automated machine gun?” with problem sets carefully crafted to lead us into “the usual suspects” failure modes. It was maybe 1/2 about learning FORTRAN and 1/2 about learning why computers are not to be trusted blindly and how you WILL screw it up, often, and what to do to fix that.

    My second class was ALGOL, which I also really loved. Block structured granddaddy of them all. Unfortunately fallen from use just about everywhere. Pascal tried to replace it with a straightjacket and syntax UberCops, and mostly just succeeded in killing it. C came along and returned the “freedom of expression” but at the expense of being a bit cryptic symbolic and with really crappy fixed format handling. (The ALGOL on the Burroughs B6700 used a format system almost identical to the FORTRAN one. Since ALGOL had the same failure as so many other languages where the I/O is left as an exercise for the implementer, the Burroughs folks could just graft that on. IMHO for any language the I/O methodology needs to have a defined and implemented method for both FIXED format and variable format data… C has a similar, though smaller, failing in that… I have a copy of an ALGOL compiler source code and would likely have installed it, but for the issue of making the I/O system “proper”…)

    In the end, C has won. Not so much for being superior to ALGOL, but for being more ubiquitous. Unix did that. At least it does have most of what was valuable in ALGOL, even if a little more cryptic, and has a lot more “bit twiddling” features. But damn it, it could sure use a fixed format READ and FORMAT statements! Despite current “style” disparaging them, they are widely used and quite useful. I really don’t think it adds anything to need to strip commas out of CSV files all the time… and put them back in each time you want a data dump / handoff.

    I would likely have done my analytic code bits in C if those statements existed. As it was, it was just a lot easier to pick up the FORTRAN FORMAT statements in GIStemp code and carry them forward into my code and do it in FORTRAN. Yeah, I’ve done fixed format data read / write in C, and while very ‘doable’ it is still a bit of a PITA IMHO. Suck in a byte stream and then parse it yourself? Why?

    BTW, I also did a decade+ as a DBA using “4th generation non-procedural languages” along with program interfaces (and the odd bit of PL1 a monster of a language in which you could write COBOL, FORTRAN, Pascal and a couple of other language styles and still have it work…) so it’s not like I’m against “modern” methods. I just find different tools best for different things. Tending to avoid the Language Wars and not really having a “favorite” ( just like I don’t have a favorite between screwdrivers and wrenches… just different best uses.)

    Ah, well, back to work ;-)

  5. E.M.Smith says:

    Well I’ll be damned:

    root@RaPiM2:/home/pi# apt-get install algol68g
    Reading package lists… Done
    Building dependency tree
    Reading state information… Done
    The following NEW packages will be installed:
    algol68g
    0 upgraded, 1 newly installed, 0 to remove and 0 not upgraded.
    Need to get 422 kB of archives.
    After this operation, 1,030 kB of additional disk space will be used.
    Get:1 http://mirrordirector.raspbian.org/raspbian/ wheezy/main algol68g armhf 2.4.1-1 [422 kB]
    0% [1 algol68g 0 B/422 kB 0%]
    Fetched 422 kB in 10s (41.1 kB/s)
    Selecting previously unselected package algol68g.
    (Reading database … 89917 files and directories currently installed.)
    Unpacking algol68g (from …/algol68g_2.4.1-1_armhf.deb) …
    Processing triggers for man-db …
    Setting up algol68g (2.4.1-1) …
    root@RaPiM2:/home/pi#

    https://packages.debian.org/sid/algol68g

    Package: algol68g (2.8-2)

    Implementation of Algol 68 as defined by the Revised Report

    Algol 68 is a secure, expression-oriented programming language with which you elegantly code algorithms without having to bother too much about irrelevant technical details and limitations inherent to many other languages.

    Algol 68 Genie offers for example: » many runtime checks facilitating debugging and improving the » reliability of your programs, native support for arbitrary precision » arithmetic including complex numbers, syntactic constructions to » support linear algebra, a gdb-style debugger and a pretty-printer to » beautify source code, optional linkage to GNU plotutils, the GNU » scientific library or PostgreSQL, extensions as UNIX pipes, regular » expression matching, and web page content fetching.

    Welcome “back to the future” with a great 1968 ALGOL Standard compiler with quasi-modern extensions…

    Questions:

    1) Who in the world is still using this?

    2) Why? Is it all just folks mired in nostalgia for ALGOL like me?

    3) Do I remember how to actually write a program in ALGOL any more?

    4) Do I really care enough to find out the answer to #3?

    Inquiring minds are wondering if those questions are worth the time…

    OTOH, a nice translation of GIStemp into ALGOL for that truly “Retro Ride Feel” might have it’s charms ;-) For when FORTRAN-77 is Just TOO New!

    Somehow I’m having this Circle Of Life moment where I’m looking at FORTRAN and ALGOL from the ’60s and ’70s but doing it on a 4 core R.Pi with more total processing power than the “dual CPU B-6700” that was nearly a super computer of it’s day… and when having 2 CPUs was a state of the art (and pushing it) thing beyond what most folks could do.

    Sometimes the world is a very strange place…

    http://www.retrocomputingtasmania.com/home/projects/burroughs-b6700-mainframe

    B6700 1971 Max of 1M word (48-bit word + 4 tag bits), 6MBytes memory
    1 – 3 CPUs and 1-2 I/O Processors
    All B6500s were field upgraded to B6700 to correct engineering problems with the B6500

    So I’ve got about double the CPUs ( assuming GPU is about the same as “I/O processors”) and I’ve got 1 GB of memory instead of 6 MB, or about 166 times the memory. But minus the stack architecture… so comparing MIPS / MFLOPS might be hard…

    https://en.wikipedia.org/wiki/Burroughs_large_systems

    ALGOL

    The Burroughs large systems implement an ALGOL-derived stack architecture, unlike linear architectures such as PDP-11, Motorola M68k, and Itanium or segmented architectures such as x86 and Texas Instruments. (This refers to the layout of the memory and how a program uses it.)

    While B5000 was specifically designed to support ALGOL, this was only a starting point. Other business-oriented languages such as COBOL were also well supported, most notably by the powerful string operators which were included for the development of fast compilers.

    The ALGOL used on the B5000 is an extended ALGOL subset. It includes powerful string manipulation instructions but excludes certain ALGOL constructs, notably unspecified formal parameters. A DEFINE mechanism serves a similar purpose to the #defines found in C, but is fully integrated into the language rather than being a preprocessor. The EVENT data type facilitates coordination between processes, and ON FAULT blocks enable handling program faults.

    The user level of ALGOL does not include many of the insecure constructs needed by the operating system and other system software. Two levels of language extensions provide the additional constructs: ESPOL and NEWP for writing the MCP and closely related software, and DCALGOL and DMALGOL to provide more specific extensions for specific kinds of system software.

    The extended version with “insecure constructs” was used to write the systems code. As the whole thing was stack oriented, it was damn fast for the day. IMHO everything from Forth to Java owe a bit to that… but on real hardware instead of a virtual machine…

    And yes, clearly I didn’t make it “back to work” after the last comment… but “this time for sure!”
    ;-)

  6. Another Ian says:

    E.M.

    O/T

    “pat

    July 30, 2015 at 6:46 pm · Reply

    posted on WUWT Tips & Notes by Jim Sawhill on 29 July:

    Pierre Gosselin at NoTricksZone (http://notrickszone.com/) is in peril of quitting due to WordPress issues.
    “Right now it’s all garbage and I’m getting ready to quit. If anyone knows a good WordPress repairman, let me know.”

    Anybody help?
    Most of his links are down. I don’t have contact info and that link is dead too.

    He and his site are an important source of material out of Europe – Climate news from Germany in English – by Pierre L. Gosselin”

    http://joannenova.com.au/2015/07/less-than-half-of-climate-scientists-agree-with-the-ipcc-95-certainty/#comment-1731266

  7. E.M.Smith says:

    @Another Ian:

    Better placed in “Tips” but “no worries”…

    Golly, I wonder what issues he is having… For me, nothing has changed. (Though I do avoid using the “beep beep boop” editor an all the other “new” garbage… )

    I’m happy to help Pierre in any way I can. Feel free to drop my email address on him. (But I’m not sure just what all I could do… I’m just a user of WordPress, not a maintainer…)

  8. E.M.Smith says:

    @Another Ian:

    A “quick look” gives me an idea. I ‘hit a couple of links’ and all of them gave me something like:

    Error 404!

    /2015/07/29/new-social-sciences-paper-shows-climate-skepticism-in-germany-prominent-in-public-opinion-polls/
    File Not Found!

    Notice that this STARTS with “2015”etc etc. No “notrickszone”. I think that the “issue” is that his domain name is not mapping to his wordpress provider. (so not substituting “wordpress.whatever.something/2015/…” for “notrickszone.”

    The HTTP connection link says
    “http://notrickszone.com/2015/07/29/new-social-sciences-paper-shows-climate-skepticism-in-germany-prominent-in-public-opinion-polls/#respond”

    so the “http://notrickszone.com” is being left off the referral link.

    I’m not having any issues as I don’t have ‘my own domain name’ being substituted.

    As the “contact” link is similarly broken, I can’t just email this to him. Please feel free to forward it if you have a method. It’s the “custom domain name” to “wordpress provider” part that is broken and likely is a simple fix for whoever provides the mapping from custom domain names to the wordpress servers.

  9. larrygeiger says:

    “Pascal tried to replace it with a straightjacket and syntax UberCops, and mostly just succeeded in killing it.” No.

    Pascal was meant to be a teaching language. As such, it was useful for many things but it was never intended to be comprehensive. Pascal has a very regular syntax that CS students can write a parser and compiler for it in just one semester. We wrote ours in Turbo Pascal.

    Pascal was initially meant to be as context-free as possible, with very few context-sensitive features. This makes creating parsers and compilers more straightforward. Pascal was also initially very strongly typed.

    I have watched as students have chosen languages such C or C++ as their language for their compiler projects and then crashed and burned miserably. Some to never recover. I personally think that it’s more important to get the whole thing working (at a basic level) and then being able to play with it than working on something for months and months and ending up with a few parts that are never complete. IMHO. YMMV.

  10. E.M.Smith says:

    @LarryGeiger:

    Ok, ok…. so I ought to have said “The folks who learned Pascal in classes ” tried to replace….

    But clearly Niklaus Wirth saw it as a follow-on from Algol and to address some aspects he thought it would do better. Those included compilers and teaching, but also many other things including operating systems… (And when I learned Pascal it was strongly typed… are you saying it isn’t so much anymore? Yeah!!! Straight jacket is off?!?!!)

    But in the quote below note the TWO intents. Systems programming (just like Burroughs had to extend Algol for that) AND teaching:

    http://www.swissdelphicenter.ch/en/niklauswirth.php

    Pascal, 1968-1972
    Freed from the constraining influence of a working group’s consensus, Wirth developped the language Pascal in Zurich. The basis was Algol-W and the desire to have a language that would satisfy the requirements of system design (compilers, operating systems, etc.). Also, there was to be a basis of clear concepts and structures, definable axiomatically and independently of any particular computer, as the language was to be suitable also for teaching in an academic environment. Pascal has satisfied these requirements; it is today one of the most widely used languages in computer science education. The first Pascal compiler was designed in Zurich for the CDC 6000 computer family, and it became operational in 1970. Already in 1972 Pascal was used in introductory programming courses.

    […]
    Abstract
    The programming language Pascal was designed in 1969 in the spirit of Algol 60 with a concisely defined syntax representing the paradigm of structured programming. Seven years later, with the advent of the micro computer, it became widely known and was adopted in many schools and universitites. In 1979 it was followed by Modula-2 which catered for the needs of modular programming in teams. This was achieved by the module construct and the separate compilation facility. In an effort to reduce language complexity and to accommodate object-oriented programming, Oberon was designed in 1988. Here we present some aspects of the evolution of this family of programming languages

    He also clearly intended it to be an improvement on Algol both for teaching and for practical use.

    1. Structured Programming and Pascal
    Pascal was born in 1969 out of an act of liberation [0]. In more than one sense. Confronted with the duty to teach programming, I had been faced with the dire options of Fortran and Algol. The former did not appeal to my taste as a scientist, the latter not to those of the practical engineer. I liberated myself from this jail by designing Pascal, convinced that an elegant style and an effective implementation were not mutually exclusive. I felt strongly — and still do — that a language used in teaching must display some style, elegance, consistency, while at the same time also reflecting the needs (but not necessarily bad habits) of practice. I wanted to design a language for both my classroom and my “software factory”.

    The second alluded liberation was from the design constraint imposed by committee work. In 1966, Algol W [1] had been a compromise bowing to many divergent opinions and requests from both an Algol committee and an Algol community. Surely, many of them were inspiring and beneficial, but others were incompatible and hindering. Some members had high ambitions of creating a language with novel features whose consequences were to be the subject of further research, whereas I had been brought up as an engineer feeling uneasy with proposals whose realization was still the subject of speculation. I wanted to have at least a concrete idea of how a construct was to be represented on available computers, and these, let me add, were rather ill-suited for any feature not already present in Fortran.

    The general idea dominating the design of Pascal was to provide a language appealing to systematic thinking, mirroring conventional mathematical notation, satisfying the needs of practical programming, and encouraging a structured approach. The rules governing the language should be intuitive and simple, and freely combinable. For example, if x+y stands for an expression, x+y should be usable as a sub expression, in assignments, as procedure parameter, or as index. For example, if a widely established convention interprets x-y-z to mean (x-y)-z, we should not redefine this to denote x-(y-z). Or if x=y is used for centuries to denote equality of x and y, we should refrain from the arrogance of replacing it by x==y. Clearly, Pascal was to build up on the notational grounds established by mathematics and Algol. Pascal and its successors were therefore called Algol-like.

    Here we see stated that the intent was to use Pascal for systems programming rather like an extended Algol had been used on the Burroughs machines that is called “ill-fated” and a resource hog. To me, this implies an intent to make something better to replace it…

    Today, it is hard to imagine the circumstances prevailing in the 1960s. We must recall that the computing community was strictly split into two professional camps. The scientists and engineers used Fortran for their programming large-scale, word-oriented, binary computers, wheres the business community used Cobol for their smaller, character-oriented, decimal machines. System programmers were labouring within computer companies using proprietary machine-code assemblers. There were attempts to unite the two worlds, such as the highly innovative Burroughs B-5000 computer, or IBM’s programming language PL/I. Both were ill-fated and devoured considerable budgets. Pascal was another such attempt, although less ambitious and without budget or industrial support. It applied the idea of recursively defined structures not only to executable statements, but also to data types. As elements, it adopted arrays (vectors, matrices) from Fortran and Algol, as well as records and files from Cobol. It allowed them to be freely combined and nested.

    And here we see concerns about practical aspects and the desire to replace the older languages (in this case citing FORTRAN) being considered on practical grounds. Not exactly a teaching ideas concern…

    The other fact about the 1960s that is difficult to imagine today is the scarcity of computing resources. Computers with more than 8K of memory words and less than 10us for the execution of an instruction were called super-computers. No wonder it was mandatory for the compiler of a new language to generate at least equally dense and efficient code as its Fortran competitor. Every instruction counted, and, for example, generating sophisticated subroutine calls catering to hardly ever used recursion was considered an academic pastime. Index checking at run-time was judged to be a superfluous luxury. In this context, it was hard if not hopeless to compete against highly optimized Fortran compilers.

    Yet, computing power grew with each year, and with it the demands on software and on programmers. Repeated failures and blunders of industrial products revealed the inherent difficulties of intellectually mastering the ever increasing complexity of the new artefacts. The only solution lay in structuring programs, to let the programmer ignore the internal details of the pieces when assembling them into a larger whole. This school of thought was called Structured Programming [2], and Pascal was designed explicitly to support this discipline. Its foundations reached far deeper than simply “programming without go to statements” as some people believed. It is more closely related to the top-down approach to problem solving.

    And a bit further on, we have concerns about “the customer”… again, not exactly focused on academia and “the student”…

    Besides structured statements, the concept of data types characterized Pascal profoundly. It implies that every object, be it a constant, a variable, a function, or a parameter has a type. Data typing introduces redundancy, and this redundancy can be used to detect inconsistencies, that is, errors. If the type of all objects can be determined by merely reading the program text, that is, without executing the program, then the type is called static, and checking can be performed by the compiler. Surely errors detected by the compiler are harmless and cheap compared to those detected during program execution in the field, by the customer. Thus static typing became an important concept in software engineering, the discipline emerging in the 1970s coping with the construction of large software complexes.

    We also have an express concern about the rate of conversion of Algol users (“Algolites”) to Pascal. To me, this implies a desire to have converted them, but that is my speculation. Still… it sure looks like…

    Yet, Pascal also suffered from certain deficiencies, more or less significant depending on personal perception and application. One of them had its origin in a too dogmatic interpretation of static typing, requiring that the type of every procedure parameter be known at compile-time. Since this included index bounds in the case of array types, the frequently convenient dynamic arrays were excluded. In hindsight, this rigidity was silly and kept many Algolites from adopting Pascal. Arrays are typically passed by a reference, and for dynamic arrays only the array bounds must be added to this information. The limited additional complexity of the compiler would certainly have been outweighed by the gained language flexibility.

    Certain other deficiencies were due to the author’s lack of courage to throw some rules inherited from Algol over board, in fear of antagonizing influential Algol programmers.
    The prime entry in this list is the famed go to statement, retained although, in principle, always replaceable by an if, while, or repeat construct. Another retained mistake was the lack of full type specification of parameters of formal procedures, through which, in principle, the entire type system could be undermined. This is illustrated by the following condensed, artificial example. Incidentally, it may also serve as an example of programming puzzles popular at the time.

    So all in all, I’ll stand by my snarky short hand of “Pascal tried to replace” [Algol], even if really it was not Pascal, but the designer of it and the users of it that did the desiring…

    That it was intended to be dual purpose and used in teaching as well is quite true, so yes, it was intended as a teaching tool and to be efficient to compile (though the above looks like that was as much due to hardware efficiency as anything else…)

    BTW, when you have a student who wants to make a C compiler, why not have them do one for TinyC? I have a fairly slim book on it that shows it’s easy to compile and not quite a toy language… Hmmmm….. All the stuff I find on line is about a fully featured C compiler:
    http://bellard.org/tcc/tcc-doc.html

    very small, but not the cut down language in the book I have from a couple of decades back… Ah… found it… it is “tiny-C”
    http://fixunix.com/cp-m/239461-tiny-c-print.html that talks about it and where some of the files can be found.

    The book is from 1985 ISBN 0-8306-1895-3 and has full source code for an interpreter and everything else needed. As it says on page 60 for facts about the PC version:

    It consists of:

    11 source files
    18 global variables
    38 functions
    1446 lines of C
    2220 bytes of data
    19,249 bytes of executable code

    I have to think a compiler would not be much different.

    IFF you are really interested and can’t find a copy of the book, I’m willing to make copies of some of mine available ( provided you don’t mind photos in your email…) It is 153 pages all told and I’d rather not do all of them so would limit it to just the ones with significant content, like the actual interpreter code and support source files…

    Oh, wait, looks like you can get it from Amazon:

    http://www.amazon.com/Learning-C-Tiny-Scott-Guthery/dp/0830618953

    6 Used from $6.45
    2 New from $2,427.01

    YIKES! Makes me sorry I actually read mine… I could use an extra $2420…

    It would be much easier to do than standard C. (Then again, depending on level, maybe too easy…)

  11. Pingback: Personal TeraFlop Climate Simulator for How Much? | Musings from the Chiefio

Comments are closed.