Step2 Overview and Sizes

STEP2 – FORTRAN zonalizing


First up, what does gistemp.txt say:


And what does the gistemp.txt file have to say about STEP2?

Step 2 : Splitting into zonal sections and homogeneization (
To speed up processing, Ts.txt is converted to a binary file and split
into 6 files, each covering a latitudinal zone of a width of 30 degrees.

The goal of the homogeneization effort is to avoid any impact (warming
or cooling) of the changing environment that some stations experienced
by changing the long term trend of any non-rural station to match the
long term trend of their rural neighbors, while retaining the short term
monthly and annual variations. If no such neighbors exist, the station is
completely dropped, if the rural records are shorter, part of the
non-rural record is dropped.

Result: Ts.GHCN.CL.1-6 – before peri-urban adjustment
Ts.GHCN.CL.PA.1-6 – after peri-urban adjustment


General Description of Function


As described in the gistemp.txt file, thisis the step that does the homoginization. We have another set of binary conversions and yet more lost of low end bits from further math. Anomalization begins here.


Sizes and Listings


Lets look inside STEP2 for how big things are. Files ending in “f” are FORTRAN program source code while those ending in “.sh” are Unix / Linux shell scripts. Oh, and “*” is the “wildcard character” that says “match anything” so *smith would match “Goldsmith” and “Tinsmith” and… So just taking a look at the line count numbers in the text files (cd STEP2, wc -l */*) :


So just how big is STEP2? This is the result of (cd STEP2; du *):



There are four shell scripts, the file which runs the programs in order (after linking all the input-files to similar names in the STEP2 directory… At the end, it moves work files and output files from the STEP2 directory into their respective directories as well.)

The work_files and to_next_step directories are empty.

Here we have wc * for all the other program files at the STEP2 level:




Taking a look at the top level script


if [[ $# -ne 1 ]]
then echo “Usage: $0 last_year_with_data (20??) ”
exit ; fi

if [[ $FC = ” ]]
then echo “set an environment variable FC to the fortran_compile_command like f90”
echo “or do all compilation first and comment the compilation lines”

infile=to_next_step/Ts.txt ; label=’GHCN.CL’ ; last_year=$1
if [[ ! -s $infile ]]
then echo “$infile not found – get it from the previous step – no action”
exit ; fi

echo “converting text to binary file”
ln -s input_files/v2.inv .
rm -f Ts.txt ; ln -s $infile Ts.txt
${fortran_compile} text_to_binary.f -o text_to_binary.exe ; text_to_binary.exe $last_year

echo “breaking up Ts.bin into 6 zonal files”
${fortran_compile} split_binary.f -o split_binary.exe ; split_binary.exe

echo “trimming Ts.bin1-6”
${fortran_compile} trim_binary.f -o trim_binary.exe ; trim_binary.exe

for i in 1 2 3 4 5 6
do mv Ts.bin${i}.trim Ts.${label}.$i

rm -f Ts.bin Ts.txt

echo “preparations for urban adjustment”
${fortran_compile} invnt.f -o invnt.exe ; ./invnt.exe Ts.${label} > Ts.${label}.station.list

echo “Creating annual anomalies ANN.d$1.[1-6]”
./toANNanom Ts.${label}

${fortran_compile} PApars.f tr2.f t2fit.f -o PApars.exe
${fortran_compile} flags.f -o flags.exe
${fortran_compile} padjust.f -o padjust.exe

./PApars ${label}
./padjust Ts.${label}

./invnt.exe Ts.${label}.PA > Ts.${label}.PA.station.list

mv ANN* Ts*bin* *exe *log* *use* work_files/.
mv Ts* to_next_step/. ; mv *list work_files/.
rm -f v2.inv

echo ; echo “created Ts.${label}.* files”
echo “move them from STEP2/to_next_step to STEP3/to_next_step”
echo “and execute in STEP3 ”

End of Script


Other than that the script seems to look for it’s input file ($1) v2.mean.comb in the directory “to_next_step”, we see what looks like a fairly straight forward process. Why is an input file in the “to_next_step” directory? “Why, don’t ask why, down that path lies insanity and ruin. -emsmith.”

About E.M.Smith

A technical managerial sort interested in things from Stonehenge to computer science. My present "hot buttons' are the mythology of Climate Change and ancient metrology; but things change...
This entry was posted in GISStemp Technical and Source Code and tagged , , , , , , . Bookmark the permalink.