GIStemp STEP4 – the process

This is step 4

STEP4 Overview

STEP4 does an update of the SBBX.HadR2 file. OK, I can live with that. And it does it via the do_comb_step4.sh top level controlling script. That works with the established pattern too.

Then comes the ringer. The same SBBX.HadR2 file is also updated via another top level script (called by nothing else, so hand run?) and via a different program. I suspect that this is an undocumented “feature” or process. So I’m going to conjoin them both in this description.
 

Sizes and Listings

 

Lets look inside STEP4 for how big things are. Files ending in “.f” are FORTRAN program source code (that when compiled into something the computer can run gets the “.exe” executable binary suffix) while those ending in “.sh” are Unix / Linux shell scripts. Oh, and “*” is the “wildcard character” that says “match anything” so *smith would match “Goldsmith” and “Tinsmith” and…

The first thing we notice looking in the STEP4_5 directory is that it has no work_files or to_next_step files subdirector. It is clear why there is no to_next_step directory (at least for the 5 portion) but the lack of a work_files directory is a bit curious… But what is in input_files this time?

cd input_files
ls -l
total 2576
-rw-r--r--   1     21875 Sep 21  1995 SBBX_LtSN.LnWE.dat.gz
-rw-r--r--   1   1291146 Jul 12  2003 oisstv2_mod4.clim.gz

So we have a couple of gzipped files… More to be sorted out later. Hopefully the scripts do any needed unzipping; but from what I can tell so far, they must be gunzipped by hand. Another undocumented feature?

So just taking a look at the line count numbers in the text files (cd STEP4_5, wc -l *) selected items :

   Lines   Words   Bytes File Name

     147     389    4216 convert.HadR2_mod4.upto15full_yrs.f
     147     396    4165 convert1.HadR2_mod4.f
       8      65     387 do.mult_year.TocnHR2.upd
      27     138     880 do_comb_step4.sh
     329                 total

 

For STEP4 there are about 329 lines of code in total (including the presumed STEP4 code of the odd multi year variations). This is just a quick first pass review since I’m really still back decoding STEP1 still.

 

Scripts:

 

Note – all FORTRAN is compiled to runnable binaries in-line in the scripts, executed, then the runnable binaries deleted, most of the time.  The multi year variation here does no such housekeeping. You would expect the human readable source code to be in a distinct archive, compiled to runnable binaries in a distinct program library once, and only deleted when a newer version had passed a Quality Assurance test.

One comment on style. I always learned that you created and used your work files in a work file directory; this code creates and uses them in the same directory as the program source code, then when done, sometimes moves them into a work directory. Not what you would expect.

There are two scripts. They both look like top level controlling scripts. I presume that do_comb_step4.sh is the “standard” one but that the script do.mult_year.TocnHR2.upd can be run for special occasions… Both will be reviewed inline here.

do_comb_step4.sh

This is the start. It runs the show.

A Deeper Look at the Top Level Script, or Scripts

 

We will take a brief look at the top level controlling script do_comb_step4.sh and, after the listing of it, I’ll describe what it does.

Begin listing of “do_comb_step4.sh”:

if [[ $# -lt 2 ]]
then echo "Usage: $0 year month1 month2";
     echo "       to add some months to SBBX.HadR2"
     exit; fi

fortran_compile=$FC
if [[ $FC = '' ]]
then echo "set an environment variable FC to the fortran_compile_command like f90"
     echo "or do all compilation first and comment the compilation lines"
     exit
fi

if [[ ! -s input_files/SBBX.HadR2 ]]
then echo "file input_files/SBBX.HadR2 not found" ; exit ; fi

mo=$2 ; if [[ mo -lt 10 ]] ; then mo='0'${mo} ; fi
if [[ ! -s input_files/oiv2mon.$1$mo ]]
then echo "file input_files/oiv2mon.$1$mo not found" ; exit ; fi

mo2=$2 ; if [[ $# -gt 2 ]]; then mo2=$3 ; fi

$FC convert1.HadR2_mod4.f -o convert1.HadR2_mod4.exe
convert1.HadR2_mod4.exe $1 $2 $mo2
trimSBBX SBBX.HadR2.upd

echo 'If all went ok, type "mv SBBX.HadR2.upd.trim input_files/SBBX.HadR2"'
echo '          then  type: do_comb_step5.sh 100 0'

 
End of script.

This scripts is simple Bourne shell. It starts off with a check that it has at least 2 parameter passed, then notifies you how to run it with the 3 passed parameters it expects. “Why? Don’t ask why, down that path lies insanity and ruin. -e.m.smith” I’d have thought you would have that memorized by now…

It then procedes to check that the environment variable “FC” is set to your FORTRAN compiler. Interestingly enough, it suggest hand editing the script to comment out the compilation steps if you want to do everything by hand. Have these people never heard of using “#DEFINE” or passed parameters to control script execution paths?

Next up, we check that the input file input_files/SBBX.HadR2 exists.

(Which it doesn’t right now. STEP3 at the end says to create it or use STEP4 code to update it. It looks like the creation of the file is left as an exercise for the student.)

The second passed parameter is assigned to “mo” and this is manipulated. We prepend an “0” on any single digits, then look for an input file of the form input_files/oiv2mon.[year][start_month]

(which also doesn’t exist right now so is also left as an exercise for the student. Maybe malt liquer would help… or Scotch… yeah, a nice single malt… Sorry, the mind wanders at times like this when faced with crap like this.)

If we have a third parameter, we assign it to “mo2” though no guidance is given as to when you would want 2 vs 3 parameters or what choices might make the anomalies work out “best”. More play time for the students…

Finally, we get to the meat of it. We compile and run the FORTRAN program convert1.HadR2_mod4.f passing to it the parameters year, monthstart and monthend that were passed into the script, or using the monthstart as the monthend if only one month was given.

At the end we are told what to do as a manual step “if all went well”. One is left to wonder how one knows if all went well, and what the acceptance criteria might be…

mv SBBX.HadR2.upd.trim input_files/SBBX.HadR2"
then  type: do_comb_step5.sh 100 0'

This just looks like a “hand tool” to automate the updating of a data set in some small fashion, but still leave lots of room for individual tuning of exactly what months get updated when. It certainly is not what I would consider to be a production quality process. Period.

OK. Some information on how to ‘create/update’ it would be nice. I guess I’m going fishing again out on the internet or in the source code or… I’d guess from the name that it’s a sea surface set of data from Hadley. Go fish.

We will take a brief look at the top level controlling script do_comb_step4.sh and, after the listing of it, I’ll describe what it does.

Begin listing of “do.mult_year.TocnHR2.upd”:

if [[ $# -lt 2 ]]; then echo "Usage: $0 year1 year2"; exit; fi
y1=$1 ; y2=$2 ; y2m=$(( y1+14 ))
if [[ y2 -lt y1 ]] ; then echo need yr1=$y1 '<' yr2=$y1 ; exit ; fi
if [[ y2 -gt y2m ]] ; then echo need yr2=$y2 '<' ${y2m} ; exit ; fi

# f90 convert.HadR2_mod4.upto15full_yrs.f -o convert.HadR2_mod4.upto15full_yrs.exe
convert.HadR2_mod4.upto15full_yrs.exe  $y1 $y2
trimSBBX SBBX.HadR2.upd

 
End of script.

Not a whole lot to work with here. Looks like a bare bones hand tool for experimentation at best. We look to make sure we have at least 2 years as passed parameters; if not we exit. If so, we pack them into the variables y1, y2 and calculate a y2m as year one plus 14.

Some basic bounds checking follows:

If our end year precedes our start year, we complain and exit.

The maximum years ahead a given year might go looking for information must be less than 14 If you are more than 14 years apart (15 full years), you get dumped.

Then we explicitly call the f90 compiler (guess now we know this part was written in the last decade and a half) and ignore the compiler environment variable (looking more an more like an experimental hand tool) to compile and run convert.HadR2_mod4.upto15full_yrs.f  with the start and end years as passed parameters.

Finally, the script trimSBBX is run with SBBX.HadR2.upd as the passed parameter.

At the end, nothing cleans up the left over binary executable nor any workfiles that might exist. There is no guidance given to the operator.

This is at best a hand operated job of a script and at worst a leftover plaything. I’d ignore it unless you really need to play with multiyear anomaly impacts (or can figure out what it’s for).

Now, we need to take a tiny peek at what FORTRAN programs are laying about and what they do.

FORTRAN source:

 

There is just one FORTRAN program called by do_comb_step4.sh:

convert1.HadR2_mod4.f

It doesn’t have much of a header:

c   To read the  oi SST version 2
c    and the masks.
c
c
      parameter (im=360,jm=180,ioff=im/2)
      parameter (iyrbeg=1880,iyreo=3000, nymax=15)
      parameter (monx=12*(iyreo-iyrbeg+1))
c
c   i is longitude index (values 1 to im)
c   j is the latitude index (values 1 to jm)

Any further depth will have to wait on the exploration of the code itself.

So what about the “other white meat” second version?

There is just one FORTRAN program called by do.mult_year.TocnHR2.upd:

convert.HadR2_mod4.upto15full_yrs.f

It doesn’t have much of a header either:

c   To read the  oi SST version 2
c    and the masks.
c
c
      parameter (im=360,jm=180,ioff=im/2)
      parameter (iyrbeg=1880,iyreo=3003, nymax=15)
      parameter (monx=12*(iyreo-iyrbeg+1))
c
c   i is longitude index (values 1 to im)
c   j is the latitude index (values 1 to jm)

Gee, these look similar… he ponders a “diff”: The diff shows them to be different in a few dozen lines, but clearly related. It looks like one was made from the other one. My guess is that the ‘multi’ year one was created as an extension of the base version. Added detail must await the detailed code review.

And that concludes the overview of STEP4 FORTRAN.

Their is no STEP4 “readme” file to reproduced below:

Advertisements

About E.M.Smith

A technical managerial sort interested in things from Stonehenge to computer science. My present "hot buttons' are the mythology of Climate Change and ancient metrology; but things change...
This entry was posted in GISStemp Technical and Source Code and tagged , , , , , . Bookmark the permalink.