I’ve spent a chunk of today with my brain in FORTRAN land. Revisiting one of my older programs.
It is one that just takes a given country code and finds the average temperature for the reporting stations in that country, by year.
I’ve run it before on GHCN v2 data to get some kind of ‘feel’ for what is happening in any given country in the data. Yes, it “has theoretical issues” in that for a country like the USA if you compare 1800 with 2010 there are a lot more stations in the hot desert southwest now, then there were in 1800. It makes no attempt at gridding. It is to visualize the ‘shape of the data’ not to make grand pronouncements about some theoretical actual temperature.
Though, as I’ve pointed out many times, “average temperature” is a fiction that has no basis in proper science. The temperature is an intrinsic property of ONE object. Take two different object and you can average the numbers, but lose the meaning. (Even a single Stevenson Screen is measuring two different bodies of air. 20 C and 100% humidity is a far different heat content from 20 C and 20% humidity…)
My favorite example of this is: Take 2 pots of water. One at 0 C the other at 20 C. The average of their temperatures is 10 C. Mix the two pots of water. What is the final temperature? The question can not be answered. THAT is the problem. What are the relative masses of water in the two pots? Was that 0 C water freshly melted, or still ice?
So my averaging temperatures is no worse than any games GIStemp or HADcrut do when they average things. And, for the inevitable “but they use anomalies!!”… um, no, they don’t. Not until very late in the process. GIStemp carries temperatures AS temperatures until near the end. Then it makes “grid box anomalies”, not individual thermometer anomalies. IMHO that is the heart of their bogus numbers. See the dT/dt category at the right side of this page for a much better way. Make anomalies FIRST. Though even it is using the “monthly average” that is made by averaging the daily high / low and then averaging that over a month – plus a boatload of adjustments and other “QA” that, yes, uses temperature averages. It’s all just a fraud of mathematics that has no anchor in the philosophy of science. Confounding an ‘average of temperatures’ with an actual temperature and confounding temperature with heat. So what I do is no worse.
At least I admit the purpose is to see what is going on in the DATA and not make particular assertions about any temperature.
With that said…
I’ve started to compare v1 with v3. Along the way, I’ve had to pass through v2. They changed a load of country codes between v2 and v1, so comparisons are a piece of work… I’ve now got a country code map between them, and v3 is basically the same as v2.
So today I got “lmyears” and “dotemps” converted to work on v1 data. (Next up is the whole dT/dt suite). This first step lets me pick a country and just see “what changed” in the average of temperatures in any given year, and any given month.
If a place added a load of hot airport thermometers, or dropped a bunch of cold mountain ones, it WILL show up as numerical shifts between the years. ( dT/dt doesn’t have that issue, as it makes anomalies by individual thermometer, then compares them).
Now what makes this a useful tool is that when you look back at, say, 1950, you would expect the same numbers in v1 vs v2 vs v3 (modulo some very tiny ‘jitter’ due to actual fixed bits. Like a 1/10 degree or 2/10).
As my first comparison, I’ve run v1 (that now works) and compared it to v2 (that I already had). V3 is still to do, but there was a comparison of v2 to v3 already done, so for now folks could just look that up and add the two bits. Yes, “I’m working on it”, but it is going slower than I’d expected… so for now, just enjoy what we do have. The good news is that once a part, like v1, is worked out, I can knock out lots of comparisons in a hurry. So questions like “Has Australia fooled around with their history more than New Zealand?” can be readily answered. (Yes, preview of coming attractions ;-)
Some reviews of v2 vs v3:
For now, the teaser
I still have to get the reports off the Linux box and onto the Open Office PC, to do pretty reporting. The new laptop has shut off things like telnet, so I’m turning that kind of tool back on again. It all takes time. So the pretty bit is coming. But right now all I’ve got is some interesting numbers.
The first report I did was Australia.
Country Code 501 in v2 / v3 but
Country Code 513 in v1.
Some years are substantially the same. That’s a “good thing” as it lets you see that there was not some gross error of getting the country code wrong or having a variable out of place in a format statement. Other years have interesting changes. The v1 data end in 1990, so not all that much can be said about changes between 1991 and 2011. Still, the other bits have an interesting nature.
I’m just going to give the “average of temperatures for a year” and not mention individual months at this tim. Later, in particular cases if interesting, I’ll add a comment.
We’d expect these not to change much for old data, as there ought not be much you can do but say “They read the thermometer and this is what they got.” Much beyond that is more fabrication than clarification. In the very old records, that’s what we see.
Year v1 v2 1865 15.0 15.0 1866 15.1 15.1 1867 16.0 16.0 1868 16.3 16.0 1869 15.9 15.9 1870 15.3 15.8 1871 15.5 16.1 1872 15.7 16.2
So we see ‘not much’ in 1865-1869 changing. 2/10 in one year can be some trivial correction or thermometer count change. And that is what we find. In v1, there are 5 thermometers in use until 1872 when it becomes 6. For v2, there have been more thermometers found. 7 in 1865-66 and then 9. In 1870 they go to 10, and 11 in ’71 and ’72. So we can speculate that the thermometers “found” for 1871, ’72 were in generally warmer places than the prior ones.
That’s what I mean by “characterize the data”. Before putting in a lot of speculative ‘fixes’ for the data bias, see what kinds of bias might be there…
By 1900, there are many more thermometers.
v1 v2 Year count temp count temp 1900 18 17.1 71 17.0 1901 18 17.1 77 17.2 1902 18 17.3 80 17.1 1903 18 17.0 88 17.0 1904 19 16.8 88 16.7 1905 19 16.6 91 16.8
What I find most remarkable about this is the stability of the average of those readings, despite the number of instruments “found” being 4 time the original number.
Then something a bit odd starts to happen in the 1940 to 1960 range. The newer data set has a pronounced “cooler” character in that time. Many more thermometers added, so one can only wonder where they were finding them. I’ll use the 1950s as a representative sample as that avoids any question of the ‘war years’. At a casual look, I don’t see much difference. Those decades are all similar.
v1 v2 Year count temp count temp 1950 43 19.3 312 18.2 1951 73 19.9 320 18.3 1952 73 20.0 327 18.1 1953 73 19.9 330 18.1 1954 74 20.1 338 18.3 1955 75 20.0 337 18.3
That’s about 2 C of “cooling” in the 1950’s that has to be corrected out of the thermometer changes by whatever “magic sauce” is expected to “fix it”… Significantly more than the actual ‘warming’ signal that is claimed to be found. A small 25% error in the “fix” is all it would take to create “global warming” via cooling that past…
It isn’t just those decades, either. Since 1934 ish was a ‘hot year’ in the USA, it’s worth noting that it changed in Australia, too:
v1 v2 Year count temp count temp 1930 31 19.3 233 18.1 1931 32 19.3 237 17.7 1932 32 19.3 238 17.9 1933 32 19.1 239 17.7 1934 32 19.4 239 17.9 1935 32 19.3 241 17.6
Knocking about 1 C to 1.5 C out per line.
When you get to the 1980 -90 time period, near the end of v1, we still have some cooling in v2 in comparison.
v1 v2 Year count temp count temp 1985 82 20.9 468 18.6 1986 81 20.9 463 19.0 1987 80 20.9 351 19.4 1988 79 21.4 327 19.9 1989 78 20.7 320 19.3 1990 78 20.3 419 19.5
I can’t for the life of me figure out why the thermometer counts bounce around so much from year to year, this late in the history, but they do. That, IMHO, just has to jerk the results around.
But by this point, we’re down to about a 1 C to 3/4 C cooling of the data. In 2008 – 2009 v2 has averages of 19.8 and 19.5 respectively, so no real change from the mid ’80s.
So what does this all mean?
At present, not too much. It does show that the impact of thermometer change is about 4 times that of the implied “global warming” signal being sought. It does show that the changes in the data via collection swamp the changes over time in a given set. It does show that v1 is very different from v2, largely due to the changing thermometers in the set. And it does show that v2 is a ‘cooler’ set of thermometers in the past than the ones used for v1. This, IMHO, highlights the “splice problem” inherent in all these data sets. The number of thermometers are small in the past, and grow over time (up until The Great Dying in 1990… for v2, the number of thermometers drops back to just 45 in 1993 ). Given that we see whole degree sized changes in the data from thermometer count changes of even lesser degree, how can we expect that plunge from 456 in 1991 to 45 in 1993 does not create a problem?…
The count then rises slowly back to 65 in 2009 when v2 ends. So one can only wonder what places are being measured from 1993 to 2012, and are they at all related to those measured in 1953?
Frankly, other than being rather cold at the bottom of the Little Ice Age, or having most of the thermometers down in the more southern cool areas then (which they were, BTW); this mostly looks like a game of “pick your thermometers” rather than one of “measure the change of heat flow”.
So that’s your “tease” on the v1 vs v2 vs v3 work.
It is proceeding. I’ve now got the test rig all set up and the FORTRAN reading v1. Next up is load v3 and then I can skip the v2 comparisons, if desired.
At first blush, looking only at Australia, changes of thermometer count are going to be the major issue.
The two biggest technical points are that the Country Codes all get scrambled, and that the WMO minor number is a different length. So comparisons by continent ought to be fairly easy (that part of CC is mostly the same) while individual station and country are a bit more complex.
I ought to have dT/dt running on v1 and v3 in a couple of days, and then we’ll have more interesting things to compare.