Some Interesting Thermometer Lifetime Numbers

Once the data are in a database, one need not do fancy Python graphs to find out interesting things. Simple SQL queries can give interesting results.

I got to wondering “How many thermometers have a short life span?” In particular, there is a roughly 60 year climate cycle that’s known to exist. How many thermometers have a total count of data points less than 60 years? Or even less than 1/2 that at 30 years? (A half cycle, so potentially strongly trend biased by the cycle).

Thermometers With Over 30 Years Coverage

My first cut at it is a simple SQL statement to find out how many if the thermometers have over 30 years data in January and July. (Doing both can show if there is a tendency to have less coverage in one season than the other – the two numbers ought to be very close).

The SQL statement I used is:

SELECT count(T.deg_c), T.stationID FROM invent3 AS I 
       INNER JOIN temps3 as T on I.stationID=T.stationID 
       WHERE T.deg_c>-90 AND T.month='JULY' 
       GROUP BY T.stationID 
       HAVING count(T.deg_C) >30
       ORDER BY count(T.deg_C) DESC;

I don’t really need the JOIN in this report as the month, temperature data, and StationID are all in the “temp3” table; but I’ve left them in so that as I change it I can add the Station Name easily from the inventory table invent3 if anything interesting pops up.

It counts how many temperature data items there are for each StationID in the month of JULY as long as the value is over -90 C. That prevents the -999 missing data flag from being counted as a data item. Then it only puts them on the report if the count if over 30 (that is, if there are more than 30 July data items, or 30 years with a July temperature). Then prints out the count and stationID sorted into descending order on count.

This gives a few thousand records, so not something to paste in here, but the very bottom can be pasted in.

|             31 | 40371711006 |
|             31 | 42574635002 |
|             31 | 14168728000 |
|             31 | 40371868001 |
|             31 | 40371093000 |
|             31 | 14168550000 |
|             31 | 42572514004 |
|             31 | 12263624000 |
|             31 | 80099912001 |
|             31 | 50194612000 |
|             31 | 50194951001 |
|             31 | 21941523001 |
|             31 | 10266215000 |
|             31 | 40371194001 |
|             31 | 40371867002 |
|             31 | 30285289000 |
|             31 | 50194980001 |
|             31 | 40371094003 |
|             31 | 40371834001 |
|             31 | 50194666000 |
|             31 | 50194903001 |
|             31 | 20840729000 |
|             31 | 61507660000 |
|             31 | 40371897003 |
|             31 | 22341020001 |
|             31 | 40371444002 |
|             31 | 12263820000 |
|             31 | 30984673000 |
|             31 | 22231137000 |
|             31 | 11864507000 |
|             31 | 61710425001 |
|             31 | 62316270000 |
|             31 | 42572382002 |
|             31 | 50194820000 |
|             31 | 50194952001 |
|             31 | 23248830001 |
|             31 | 70089665001 |
|             31 | 20551848000 |
|             31 | 63608543000 |
|             31 | 50194527000 |
|             31 | 30285230000 |
|             31 | 40371069001 |
|             31 | 30187400001 |
|             31 | 30580346001 |
|             31 | 40371396001 |
|             31 | 14063270000 |
|             31 | 50194818000 |
|             31 | 20555696000 |
|             31 | 12061820000 |
|             31 | 50194377001 |
|             31 | 70089034000 |
|             31 | 40371911001 |
|             31 | 11965465000 |
|             31 | 12365660000 |
|             31 | 40371735006 |
|             31 | 62316066000 |
|             31 | 50194735000 |
|             31 | 20840766000 |
|             31 | 14168842001 |
|             31 | 42572383002 |
|             31 | 50194899001 |
|             31 | 41278711000 |
|             31 | 50396253000 |
|             31 | 40371510001 |
|             31 | 40371887002 |
|             31 | 10160536000 |
|             31 | 50194982001 |
|             31 | 11562464001 |
|             31 | 20556116000 |
|             31 | 40371187002 |
|             31 | 61402844001 |
|             31 | 62316300000 |
|             31 | 10764900000 |
|             31 | 40371968000 |
|             31 | 41278705000 |
|             31 | 50195666001 |
|             31 | 15260750000 |
|             31 | 30285283000 |
|             31 | 62316230000 |
|             31 | 15960035000 |
|             31 | 62316350000 |
|             31 | 42572320002 |
+----------------+-------------+
5249 rows in set (38.62 sec)

So 5249 thermometers have over 30 years of July data in them. But a fair number have only 31, or 32 years of data. Still pretty short.

MariaDB [temps]> SELECT count(stationID) FROM invent3;
+------------------+
| count(stationID) |
+------------------+
|             7280 |
+------------------+

Since there are 7280 total thermometers in the inventory, that’s 5249/7280 or about 72% of the thermometers have a minimal lifetime that could be useful.

Here’s the result for January (I won’t paste in as much of the actual report, just enough to get a flavor of how similar / different things might be):

|             31 | 62316066000 |
|             31 | 50194735000 |
|             31 | 11064753000 |
|             31 | 10160581000 |
|             31 | 50397502000 |
|             31 | 40371125002 |
|             31 | 40371887002 |
|             31 | 64813583000 |
|             31 | 21841314000 |
|             31 | 50194982001 |
|             31 | 12263772000 |
|             31 | 61402844001 |
|             31 | 20556065000 |
|             31 | 23248820000 |
|             31 | 62316350000 |
+----------------+-------------+
5197 rows in set (39.70 sec)

Only 5197. Hmmm… 5249-5197= 51 more July reports than January. Wonder if leaving out 1% more mid-winter data items might bias things?… It will be interesting to find out if more winter data is left out recently and more summer data in the past… but that also needs sorting by hemisphere. Something for later… As by far most of the thermometers are in the Northern Hemisphere, the gross comparison is still informative.

30 Years and Under Coverage

Flip it around and ask how many have under 31 years, and print them in descending order so you can see how bad it gets in the thin end:

SELECT count(T.deg_c), T.stationID FROM invent3 AS I 
       INNER JOIN temps3 as T on I.stationID=T.stationID 
       WHERE T.deg_c>-90 AND T.month='JULY' 
       GROUP BY T.stationID 
       HAVING count(T.deg_C) <31
       ORDER BY count(T.deg_C) DESC;

Note that this is not CONTIGUOUS data. Mostly it will be, but nothing prevents a station from reporting in 7 chunks of 5 years each scattered through the (roughly) 165 years and still showing up as a 35 year “record”. I know from looking at the data in detail some years ago that most stations are not too disjoint. A few are, but not many.

There’s a whole lot of truly useless stations in the data set:

|             11 | 61507283001 |
|             11 | 42570343001 |
|             11 | 40371874003 |
|             11 | 13060136001 |
|             11 | 50194862001 |
|             11 | 40371123002 |
|             11 | 50998637003 |
|             11 | 50194676000 |
|             11 | 50195664000 |
|             11 | 40371094001 |
|             11 | 13060156000 |
|             11 | 50194828000 |
|             11 | 30580219001 |
|             11 | 63834336000 |
|             11 | 50195871001 |
|             11 | 40371868000 |
|             11 | 41678806002 |
|             11 | 13060230001 |
|             11 | 40371816001 |
|             11 | 61710177000 |
|             11 | 50195979003 |
|             11 | 40371109004 |
|             11 | 50998836001 |
|             11 | 42572253004 |
|             11 | 42572765006 |
|             11 | 61710546000 |
|             11 | 40371930000 |
|             11 | 41476382001 |
|             10 | 15260714000 |
|             10 | 42572257004 |
|             10 | 21648080000 |
|             10 | 50194829000 |
|             10 | 31386545001 |
|             10 | 11562402001 |
|             10 | 42574781004 |
|             10 | 62240176002 |
|             10 | 30485767002 |
|             10 | 64308227001 |
|             10 | 70089053003 |
|             10 | 40371050001 |
|             10 | 63834344000 |
|             10 | 10664390000 |
|             10 | 15260775000 |
|             10 | 62316239000 |
|             10 | 41476595001 |
|             10 | 41678806003 |
|             10 | 11562465000 |
|             10 | 20940680001 |
|             10 | 15464246000 |
|             10 | 63512399001 |
|             10 | 40278583001 |
|             10 | 10468026000 |
|             10 | 41476692001 |
|             10 | 61111418000 |
|             10 | 40678313000 |
|             10 | 10160400001 |
|             10 | 10764900001 |
|             10 | 50397390001 |
|             10 | 70089055003 |
|             10 | 50194896000 |
|             10 | 42572469002 |
|             10 | 64917022002 |
|             10 | 20647065000 |
|             10 | 21240588001 |
|             10 | 15567541000 |
|             10 | 50194845002 |
|             10 | 31281200003 |
|             10 | 40371447003 |
|             10 | 20647046000 |
|             10 | 42570251001 |
|             10 | 70088963001 |
|             10 | 15260760001 |
|             10 | 40678368001 |
|             10 | 40278596002 |
|             10 | 10468226000 |
|             10 | 10764950003 |
|             10 | 41678795002 |
|             10 | 50793116002 |
|             10 | 40371603003 |
|             10 | 21544218001 |
|             10 | 21648088001 |
|             10 | 40678348000 |
|             10 | 41378397001 |
|             10 | 41678792000 |
|             10 | 20647005000 |
|             10 | 23659559000 |
|             10 | 62240176003 |
|             10 | 21648019000 |
|             10 | 20647070000 |
|             10 | 12462010004 |
|             10 | 43578535002 |
|             10 | 40778464001 |
|             10 | 10664390001 |
|             10 | 61710555003 |
|             10 | 15567461000 |
|             10 | 42572263001 |
|             10 | 21941506001 |
|             10 | 30580097003 |
|             10 | 20559362001 |
|             10 | 20647037000 |
|             10 | 40371855004 |
|             10 | 42572213000 |
|             10 | 15260740000 |
|             10 | 20647020000 |
|             10 | 12263737000 |
|             10 | 40371799003 |
|             10 | 15464201001 |
|             10 | 21648071000 |
|             10 | 30383827001 |
|             10 | 10764901000 |
|             10 | 50397395000 |
|             10 | 12462012002 |
|             10 | 61507149001 |
|             10 | 50194404000 |
|             10 | 80099913001 |
|             10 | 15567753000 |
|             10 | 42572438003 |
|             10 | 64502512001 |
|             10 | 20647067000 |
|             10 | 12462010001 |
|             10 | 14063240000 |
|             10 | 42572297001 |
|             10 | 31281200004 |
|             10 | 40778460004 |
|             10 | 20647050000 |
|             10 | 63512145001 |
|             10 | 40878663001 |
|             10 | 63913160000 |
|             10 | 50793116003 |
|             10 | 20647028000 |
|             10 | 12263766000 |
|             10 | 15260720000 |
|             10 | 15464224000 |
|             10 | 21648095000 |
|             10 | 30684069001 |
|             10 | 40278383001 |
|             10 | 41678793000 |
|             10 | 20647008000 |
|             10 | 12263708001 |
|             10 | 40371632001 |
|             10 | 20940623001 |
|             10 | 42572231007 |
|             10 | 30781010000 |
|             10 | 20647075000 |
|             10 | 40371050003 |
|             10 | 40371101007 |
|             10 | 20647060000 |
|             10 | 65103649001 |
|             10 | 21544325001 |
|             10 | 31281200001 |
|             10 | 40778460001 |
|             10 | 42572530002 |
|             10 | 41476743001 |
|             10 | 20647039000 |
|             10 | 52891950000 |
|             10 | 61402944001 |
|             10 | 10764950001 |
|             10 | 40371187005 |
|             10 | 42572793006 |
|             10 | 20647022000 |
|             10 | 65103334001 |
|             10 | 21648078000 |
|             10 | 41578744001 |
|             10 | 20647003000 |
|             10 | 20940604001 |
|             10 | 13761641001 |
|             10 | 62240176001 |
|             10 | 30580259002 |
|             10 | 11763473001 |
|             10 | 20647068000 |
|             10 | 14063247000 |
|             10 | 14862722000 |
|             10 | 40371878004 |
|             10 | 20448995000 |
|             10 | 40371944002 |
|             10 | 40778460005 |
|             10 | 20647052000 |
|             10 | 61816622001 |
|             10 | 50194674000 |
|             10 | 30485470001 |
|             10 | 21941782002 |
|             10 | 42572365005 |
|             10 | 63401055001 |
|             10 | 10468328000 |
|             10 | 20647031000 |
|             10 | 40371906001 |
|             10 | 50194832000 |
|             10 | 64606700001 |
|             10 | 10764931001 |
|             10 | 12263714000 |
|             10 | 43104261000 |
|             10 | 13060141001 |
|             10 | 15464180000 |
|             10 | 21648053000 |
|             10 | 20647061000 |
|             10 | 12462008003 |
|             10 | 43578526011 |
|             10 | 30580110001 |
|             10 | 20647041000 |
|             10 | 40278596001 |
|             10 | 10764950002 |
|             10 | 41678795001 |
|             10 | 50793116001 |
|              9 | 40371083002 |
|              9 | 12462016001 |
|              9 | 65103761001 |
|              9 | 15464014000 |
|              9 | 40371078003 |
|              9 | 10764931002 |
|              9 | 41678793002 |
|              9 | 61912941001 |
|              9 | 63213457000 |
|              9 | 12462012001 |
|              9 | 40371475004 |
|              9 | 42591190001 |
|              9 | 15464348000 |
|              9 | 42572257006 |
|              9 | 40371722003 |
|              9 | 40371951004 |
|              9 | 40371475001 |
|              9 | 63913266000 |
|              9 | 15464018000 |
|              9 | 40371888001 |
|              9 | 31281225001 |
|              9 | 41678795000 |
|              9 | 42572257002 |
|              9 | 11064702000 |
|              9 | 15464074000 |
|              9 | 10764880001 |
|              9 | 16268461001 |
|              9 | 40678353000 |
|              9 | 50194726001 |
|              9 | 30781100000 |
|              9 | 40371957001 |
|              9 | 42574570001 |
|              9 | 40371726004 |
|              8 | 30984628001 |
|              8 | 62906597001 |
|              8 | 42574598002 |
|              8 | 40371943007 |
|              8 | 42572773007 |
|              8 | 50194880002 |
|              8 | 40371899002 |
|              8 | 15464207000 |
|              8 | 42572409003 |
|              8 | 40371953002 |
|              8 | 30984630002 |
|              8 | 42572250002 |
|              8 | 15464315000 |
+----------------+-------------+
2031 rows in set (39.86 sec)

About 28% “junk” in terms of climate cycle 1/2 lifetime.

How about 20 years or less of data?

|              8 | 30984630002 |
|              8 | 42572250002 |
|              8 | 15464315000 |
+----------------+-------------+
1055 rows in set (39.68 sec)

So 14.5% of the thermometers in the set are basically useless for anything having to do with trends even shorter than 1/2 of the known 30 year cycles. Golly.

Let’s look at January ( I won’t paste in as much sample of the report here either). First, the less than 31 years:

|              8 | 21941506001 |
|              8 | 50194880002 |
|              8 | 41678793000 |
|              8 | 42572231007 |
|              8 | 40371814004 |
|              8 | 10764880001 |
|              8 | 40371957001 |
+----------------+-------------+
2083 rows in set (39.76 sec)

Then the less than 21 years (so 20 or less):

|              9 | 10764950002 |
|              8 | 64308227001 |
|              8 | 63213457000 |
|              8 | 12462012001 |
|              8 | 40371447003 |
|              8 | 21941506001 |
|              8 | 50194880002 |
|              8 | 41678793000 |
|              8 | 42572231007 |
|              8 | 40371814004 |
|              8 | 10764880001 |
|              8 | 40371957001 |
+----------------+-------------+
1087 rows in set (39.74 sec)

So we have 2031 vs 2083 (July / Jan) and then 1055 vs 1087. About 52 and 32 more short January thermometers than short July thermometers in the very short ranges. Think dropping a few more January thermometers will matter? GIStemp drops thermometers with too short a record when fabricating their hypothetical averages…

Here’s the SQL for that last report. You can see that all that changes is the month and the duration:

SELECT count(T.deg_c), T.stationID FROM invent3 AS I 
       INNER JOIN temps3 as T on I.stationID=T.stationID 
       WHERE T.deg_c>-90 AND T.month=' JAN' 
       GROUP BY T.stationID 
       HAVING count(T.deg_C) <21
       ORDER BY count(T.deg_C) DESC;

Thermometer Count Over 60 Years Coverage

IMHO, anything with less than a 60 year record simply is defacto biased due to there being a known sine wave of 60+ years. How many January and July thermometers are not able to detect that cycle?

Wow! For July:

|              8 | 62906597001 |
|              8 | 42574598002 |
|              8 | 40371943007 |
|              8 | 42572773007 |
|              8 | 50194880002 |
|              8 | 40371899002 |
|              8 | 15464207000 |
|              8 | 42572409003 |
|              8 | 40371953002 |
|              8 | 30984630002 |
|              8 | 42572250002 |
|              8 | 15464315000 |
+----------------+-------------+
4201 rows in set (40.87 sec)

4201 / 7280 = 57.7% of the data sources have a lifetime less than ONE known 60 year cycle. Less than 1/2 the data is usable for finding anything about climate. (Remember your Nyquist theory and sample size requirements).

Next, January:

|              9 | 40371726004 |
|              9 | 10764950002 |
|              8 | 64308227001 |
|              8 | 63213457000 |
|              8 | 12462012001 |
|              8 | 40371447003 |
|              8 | 21941506001 |
|              8 | 50194880002 |
|              8 | 41678793000 |
|              8 | 42572231007 |
|              8 | 40371814004 |
|              8 | 10764880001 |
|              8 | 40371957001 |
+----------------+-------------+
4226 rows in set (41.62 sec)

So 4226 – 4201 is “only” 25 more stations fail in January than July to have enough data… or a full 58% of the thermometers. That leaves only 42% able to detect any 60 year cycle and longer cycles hopeless.

In Conclusion

Which has me wondering… How many thermometers have 100 years or more data?

SELECT count(T.deg_c), T.stationID FROM invent3 AS I 
       INNER JOIN temps3 as T on I.stationID=T.stationID 
       WHERE T.deg_c>-90 AND T.month='JULY' 
       GROUP BY T.stationID 
       HAVING count(T.deg_C) >100
       ORDER BY count(T.deg_C) DESC;
[...]
|            101 | 21047651001 |
|            101 | 42500036253 |
|            101 | 42500420086 |
|            101 | 65103316001 |
|            101 | 42500326155 |
|            101 | 42500425477 |
|            101 | 42500308631 |
+----------------+-------------+
1552 rows in set (40.39 sec)

Golly! Just 1552 for July. That’s 21% of the thermometers.

|            101 | 63401465000 |
|            101 | 63401212000 |
|            101 | 40371630002 |
|            101 | 21047595001 |
|            101 | 42500083163 |
|            101 | 42500127482 |
|            101 | 42500218618 |
|            101 | 63401403000 |
|            101 | 42500451666 |
|            101 | 20743339000 |
|            101 | 21047588001 |
|            101 | 42500248501 |
|            101 | 42570026000 |
|            101 | 42500158714 |
|            101 | 42500214106 |
|            101 | 21135229000 |
|            101 | 42500100010 |
|            101 | 42572491003 |
|            101 | 42500294369 |
|            101 | 42500482715 |
|            101 | 42500123418 |
|            101 | 61710639001 |
|            101 | 42500058204 |
|            101 | 42500047195 |
|            101 | 42500349445 |
|            101 | 42500253035 |
|            101 | 42500417336 |
|            101 | 40371630004 |
|            101 | 42500087020 |
|            101 | 42500221865 |
|            101 | 42500456610 |
|            101 | 42500367477 |
|            101 | 21047651001 |
|            101 | 42500255310 |
|            101 | 42500246601 |
|            101 | 42500045032 |
|            101 | 21135746000 |
|            101 | 42500042319 |
+----------------+-------------+
1547 rows in set (39.80 sec)

Almost the same number for January. 1552-1547 = 5 more missing in January. Note, too, the prevalence of station numbers starting with 4 North America and then some 6 Europe and 2 Asia. Not a lot of the rest of the world had long records…

MariaDB [temps]> SELECT region, region_name FROM continent;
+--------+---------------------------+
| region | region_name               |
+--------+---------------------------+
| 1      | Africa                    |
| 2      | Asia                      |
| 3      | South America             |
| 4      | North America             |
| 5      | Australia Pacific Islands |
| 6      | Europe                    |
| 7      | Antarctica                |
| 8      | Ship Stations Ocean       |
+--------+---------------------------+
8 rows in set (0.40 sec)

If I sort in ascending order so the very longest lived records are at the end of the report, you can see just how biased the early years are toward Europe. There is simply no way to get a “Global” long term anything with this distribution (for July):

SELECT count(T.deg_c), T.stationID, I.name FROM invent3 AS I 
       INNER JOIN temps3 as T on I.stationID=T.stationID 
       WHERE T.deg_c>-90 AND T.month='JULY' 
       GROUP BY T.stationID 
       HAVING count(T.deg_C) >100
       ORDER BY count(T.deg_C);
[...]
|            160 | 20558362000 | SHANGHAI            CHINA      |
|            160 | 22223804000 | SYKTYVKAR                      |
|            160 | 61507190000 | STRASBOURG                     |
|            160 | 42572532000 | PEORIA/GREATE                  |
|            161 | 63608535000 | LISBOA/GEOF                    |
|            161 | 22228275000 | TOBOL'SK                       |
|            162 | 20742807000 | CALCUTTA/ALIP                  |
|            162 | 62004013000 | STYKKISHOLMUR                  |
|            163 | 22224959000 | JAKUTSK                        |
|            163 | 42572658002 | FORT SNILLING                  |
|            163 | 61912882000 | DEBRECEN                       |
|            164 | 63715260000 | SIBIU                          |
|            164 | 61507222000 | NANTES                         |
|            164 | 61710640001 | FRANKFURT A MAIN    W.GERMANY  |
|            164 | 65103917000 | BELFAST/ALDER                  |
|            165 | 64606660000 | ZURICH (TOWN/                  |
|            165 | 22230710000 | IRKUTSK                        |
|            165 | 22229838000 | BARNAUL                        |
|            166 | 42572290000 | SAN DIEGO/LIN                  |
|            167 | 65033846000 | MIKOLAIV                       |
|            167 | 62316239001 | ROME                ITALY      |
|            168 | 61507650000 | MARSEILLE/MARIGNANE FRANCE     |
|            168 | 64606719001 | ST. BERNARD          SWITZERLA |
|            169 | 62316059000 | TORINO/CASELL                  |
|            170 | 22228440000 | SVERDLOVSK          USSR       |
|            172 | 63401489001 | OSLO/BLINDERN                  |
|            173 | 63133815000 | KISINEV                        |
|            174 | 42572537000 | DETROIT MICHIGAN WBAS          |
|            175 | 63834880000 | ASTRAHAN'                      |
|            175 | 65033837000 | ODESA                          |
|            178 | 60311320000 | INNSBRUCK/UNIVERSITYAUSTRIA    |
|            178 | 63401098000 | VARDO                          |
|            180 | 63827612000 | MOSKVA                         |
|            180 | 42572434000 | ST.LOUIS/LAMB                  |
|            180 | 61326038000 | TALLIN                         |
|            182 | 60606447000 | UCCLE                          |
|            182 | 62103969000 | DUBLIN AIRPOR                  |
|            182 | 42572405000 | WASHINGTON/NA                  |
|            183 | 61710739000 | STUTTGART/                     |
|            185 | 61402974000 | HELSINKI/SEUTULA               |
|            185 | 62626422000 | RIGA                           |
|            189 | 42572504003 | NEW HAVEN/TWEED     USA        |
|            190 | 63827595000 | KAZAN'                         |
|            192 | 20743279000 | MADRAS/MINAMB                  |
|            195 | 65033345000 | KYIV                           |
|            196 | 63401317000 | BERGEN/FREDRIKSBERG            |
|            197 | 65103160001 | EDINBURGH/ROYAL OBS.UK         |
|            201 | 63822550000 | ARHANGEL'SK                    |
|            201 | 60311231000 | KLAGENFURT-FL                  |
|            202 | 65103779001 | GREENWICH/MARITIME MUK         |
|            211 | 61710866001 | MUNCHEN/RIEM        W.GERMANY  |
|            216 | 42572509000 | BOSTON/LOGAN                   |
|            216 | 62316080000 | MILANO/LINATE                  |
|            220 | 63512375000 | WARSZAWA-OKEC                  |
|            220 | 63401258001 | TRONDHEIM/TYHOLT    NORWAY     |
|            221 | 61912840000 | BUDAPEST/                      |
|            223 | 63512424000 | WROCLAW II                     |
|            224 | 64606642001 | BASEL/BINNINGEN     SWITZERLAN |
|            226 | 60311012000 | KREMSMUENSTER                  |
|            233 | 61206186000 | KOBENHAVN/                     |
|            235 | 62826730000 | VILNIUS                        |
|            235 | 61710962000 | HOHENPEISSENB                  |
|            236 | 61111518000 | PRAHA/RUZYNE                   |
|            239 | 64502464000 | STOCKHOLM                      |
|            241 | 60311035000 | WIEN/HOHE WAR                  |
|            244 | 61710381000 | BERLIN-DAHLEM                  |
|            244 | 61507150000 | PARIS/LE BOURGET    FRANCE     |
|            261 | 63826063000 | ST.PETERBURG                   |
|            263 | 64606700000 | GENEVE-COINTR                  |
|            291 | 61710384000 | BERLIN-TEMPEL                  |
|            310 | 63306260000 | DE BILT                        |
+----------------+-------------+--------------------------------+
1552 rows in set (40.53 sec)

And that is why they must “infill”, fabricate, homogenize, and generally make up values. They simply don’t have any actual long lifetime global data from which to find a long term global trend.

If anyone wants one of those reports above with names added, I can do that and paste it into a comment. Just let me know if you are interested in who the short stations are. For example, here’s everything under 10 years for January data:

+----------------+-------------+--------------------------------+
| count(T.deg_c) | stationID   | name                           |
+----------------+-------------+--------------------------------+
|              9 | 40371069002 | SMITH RS,AL                    |
|              9 | 42572257002 | BRYAN/AFB                      |
|              9 | 40371944002 | PINE PASS,BC                   |
|              9 | 14168242001 | MAFEKING               SOUTH A |
|              9 | 40678313000 | ISABEL RUBIO,                  |
|              9 | 42574570001 | WRIGHT PATTERSON/AFB           |
|              9 | 65103649001 | FAIRFORD                       |
|              9 | 65103761001 | GREENHAM                       |
|              9 | 11763473001 | JIGGIGA                        |
|              9 | 42591190001 | PUUNENE/CAA AIRPORT 312        |
|              9 | 12462016001 | TUMMINA/CRISPI                 |
|              9 | 41678795000 | SANTIAGO                       |
|              9 | 50194829000 | HAMILTON AIRP                  |
|              9 | 40371953002 | LOWER POST,BC                  |
|              9 | 42572469002 | BUCKLEY FIELD/ANG              |
|              9 | 15464018000 | BASOKO                         |
|              9 | 40371899002 | MASSET CFS,BC                  |
|              9 | 40371726004 | CLOVA,QU                       |
|              9 | 50194832000 | WARRNAMBOOL A                  |
|              9 | 42572257006 | FORT HOOD                      |
|              9 | 50194845002 | COLAC (ELLIMINYT)              |
|              9 | 10764950002 | SANGMELIMA          CAME       |
|              9 | 11064702000 | BOL-BERIM                      |
|              9 | 40371603003 | WESTPORT,NS                    |
|              9 | 62906597001 | ECHTERNACH                     |
|              9 | 42574781004 | VALDOSTA/MOODY AFB             |
|              9 | 31281200001 | DIRKSHOOP                      |
|              9 | 80099913001 | SHIP R                         |
|              9 | 40371083002 | CHESTERFIELD INLET,NW          |
|              9 | 41678793002 | LOS NARANJOS                   |
|              9 | 41678795002 | SANTA FE                       |
|              9 | 42572257004 | ROBERT GRAY/AAF                |
|              9 | 50194896000 | ALBURY AIRPOR                  |
|              9 | 40371817003 | BATTLE HARBOUR,NF              |
|              9 | 50194612000 | BULLSBROOK PE                  |
|              9 | 64917022002 | BUYUKDUZ                       |
|              9 | 70089055003 | DEST. NAVAL MELCHIOR           |
|              9 | 31281225001 | VICTORIA                       |
|              8 | 42572231007 | NEW ORLEANS/NAS                |
|              8 | 64308227001 | TORREJON                       |
|              8 | 41678793000 | DAVID                          |
|              8 | 50194880002 | NOOJEE                         |
|              8 | 10764880001 | TIBATI                         |
|              8 | 40371957001 | EAGLE PLAINS,YT                |
|              8 | 21941506001 | DIR                            |
|              8 | 40371814004 | LA TABATIERE,QU                |
|              8 | 63213457000 | TIVAT                          |
|              8 | 40371447003 | ELKHORN,MA                     |
|              8 | 12462012001 | BENI ULID                      |
+----------------+-------------+--------------------------------+
49 rows in set (39.95 sec)

It takes about 2 minutes to make one of these custom for a given month and duration range. Biggest issue is just they are typically so long I figure folks would be put off if I pasted the whole thing in. But if you want it, holler and I’ll paste it in ;-)

Subscribe to feed

Advertisements

About E.M.Smith

A technical managerial sort interested in things from Stonehenge to computer science. My present "hot buttons' are the mythology of Climate Change and ancient metrology; but things change...
This entry was posted in AGW Science and Background, NCDC - GHCN Issues, Tech Bits and tagged , , , , . Bookmark the permalink.

16 Responses to Some Interesting Thermometer Lifetime Numbers

  1. Graeme No.3 says:

    2 comments;
    Dublin airport has over 100 reports.
    Doesn’t seem to be anything from Australia, although that might be because the Bureau of Meteorology thinks Climate started in 1910. (Despite a Stephenson screen installed in Melbourne parkland before 1869).

  2. andysaurus says:

    Is there a way of determining the number of consecutive months figures for each station. I envisage a temporary table as a Cartesian product of years and months with months allocated their number, Jan =1, Feb =2 etc. This could then be sorted on year and numeric month and each value assigned a sequential number.
    The temporary table could then be joined to the readings and the product tested to check for the number of continuous readings. I am not a DBA, so I don’t know how to do the last bit. I’m sure there are smart people on your blog, or you yourself EM, who can work it out.

  3. H.R. says:

    E.M., you posed the question of whether or not missing Winter readings could make the record warmer. Supposedly, if anomalies are used, the seasonality is removed so it wouldn’t matter.

    What I don’t know is if any of the seasons have larger, skewed ranges than other seasons. For example (number in F), the average January temperature in my neck of the woods is about 30. The skewness and range seems to favor cold. It’s unusual to get +20 degrees but not unusual to get minus 20 degrees and common enough to get minus 30 degrees with the odd minus 40 degrees such as we’ve had three times this Winter. Winter is skewed to the cold side.

    Summer is similar with an average of 80 degrees, but the high excursions are +10 to +15 degrees with rarer 100 degree scorcher days. But the summer range to the low side is more stable than Winter. In Summer, it’s unusual to get minus 20 degrees and minus 30 degrees is pretty much unheard of. So Summer is skewed to the warm side.

    I understand the whys behind this, but I don’t know for sure how the skewness affects all those temperatures around the globe. Missing Winter records from newer thermometers would make the average anomaly temperature in my region warmer. There is more skew to cold in Winter and more skew to warm in Summer.

    What I don’t know is if it all washes out on a global basis. For example, does seasonal skewness hold in the 2-season tropics? How about the 2-1/2 season subtropics? How about towards the poles where it’s two seasons; cold and damn cold?

    Supposedly, using anomalies fixes this so those missing Winter records shouldn’t matter, but is that true if there is skewness to the readings?

    I’ll stop babbling now and go get my second cup of coffee on this lovely Sunday morning where I’ve lost an hour of sleep to Daylight Savings Time.

    BTW, where do they keep those daylight hours they saved? How come they never withdraw that extra daylight from the Daylight Savings Bank in the Winter and use it when we need the extra hours? Who is hoarding all that saved daylight? I want to have a little talk with them.

  4. andysaurus says:

    I just remembered, you assign a unique sequential number using the ‘rank’ verb in sql. I’m sure you already knew.

  5. Larry Ledwick says:

    Hmmmm that is interesting – this item caught my eye:

    ==================================================
    | 9 | 42572469002 | BUCKLEY FIELD/ANG |
    ==================================================

    Buckley field has been in essentially continuous air operations since before WWII (1938)

    From Wiki ( https://en.wikipedia.org/wiki/Buckley_Air_Force_Base )
    Buckley Air Force Base (IATA: BFK, ICAO: KBKF, FAA LID: BKF) is a United States Air Force base in Aurora, Colorado, that was established by the U.S. Army as an auxiliary field and bombing range in 1938, and activated as a designated installation in 1941. The base was named in honor of World War I Army pilot First Lieutenant John Harold Buckley.

    Buckley (Air) Field (1938 – 1946)
    Buckley Naval Air Station (1947 – 1959)
    (My dad was stationed here in post WWII at the US Navy Engine Preservation Shop putting Navy fighter aircraft engines into mothball preservation)
    Then as Buckley Air National Guard base (1960 – 2000)
    And currently as Buckley Air Force Base (2000 – date)

    I wonder how many of those short duration stations are actually the same physical location but under different controlling agencies or names?

  6. Bill in Oz says:

    EM, here where I live, the Bureau of Misinformation has records for my own town, Mt Barker, going back to 1863. Though there is a gap in the temp readinggs of about 37 years ( 1888-1921 ).. I suspect they have these but BOM thinks that thye are ‘suspect’ and so will not make them public online…

    I am not a computer geek..

    So I am perplexed by the significance of the your ‘conclusions’ for this post….
    Bill

  7. E.M.Smith says:

    @AndySaurus:

    Something I found effective is to just print out the block of temperatures for a station. I did a bunch of that back about 2009 on GHCN v2 data. You do the 12 months across and by year, so stations with a 50 year span take all of 50 lines. Missing data shows up as what spots or lines. It is it’s own graphic way of seeing the holes in the data. It would be nice to have some metric for it, though… I’ll let that idea steep a while…

    @Larry L:

    Stations are mostly keyed off of their Continent/Country and WMO number. “Name” is a dependent field. The V2 data had a “Modification flag” that indicated a different station a the same WMO site, but as I understand it, that is no homogenized away and all you get is the one “fixed up” record.

    Basically, names are not a key field.

    @Bill In Oz:

    The “conclusions” are pretty modest here and don’t need being a computer guy…

    I’m mostly just looking at how much of the data set is thermometer records that fail Nyquist Sample theory in the time domain. Put more clearly, when you have a Sine Wave of 60 years, random 30 year segments can give you an up trend, down trend, or flat. Depending on start and end dates. In a situation like the Earth, with a Bi-Polar See Saw, having such short records in the N. Hemisphere and S. H. (but offset by 30 years from each other) can give a fake “global” 60 year trend when “spiced” together (even if using the “reference station method”).

    So having a data set where only 1/5 of your thermometers have a 100 year record (only slightly long enough for the 60 year known cycles to be captured) is just shouting, no, DEMANDING a false answer due to those Nyquist failings and attempting to fit a trend line to cyclical data (both known Very Bad Things to do and both rampantly done in “climate science”). I’m pointing at the data and showing it’s crap in terms of length of station records…

  8. Bill in Oz says:

    Thanks E M. I get it !

    How does this compute with the Oz data from our BOM (Bureau of Misinformation ) ?

    Recently BOM decided that all the old temperature station readings were wrong.. And ‘homogenised’ them so they were all cooler…

    thus establishing very ‘scientifically’ that the Australian continent is warming…

  9. Kneel says:

    “How does this compute with the Oz data from our BOM (Bureau of Misinformation ) ?”

    Urk.
    Keep an eye out for Jennifer Marohasy doing a crowd funding for “independent” weather data – if this goes the way I hope, should be able to leave BoM no-where to go…
    Basically, as skeptics, we need to stop bitching about BoM etc, and start trying to help them – a crowd funded weather station done using the same gear, installed by the same contractors, but different data collection settings, such that data can be decimated to mimic BoM data, and filling “gaps” in the coverage, stations that are no longer there etc. Grab the data properly, and save it all to a database that replicates itself into a cloud server, so instant data when on-line and no missing data when it goes off-line. etc etc.
    But most importantly, it’s “the vibe” as they say – “BoM is great, lots of data etc, but alas for me, just not what I want. So here is my system, same gear, same installer, have a duplicate station to prove it matches existing stations, etc etc. Here BoM, you can add this for anyone who needs/wants it, no charge. Oh, lookit! That 36 degree maximum you measured was only there for 1 second of the whole minute, might be a problem guys… Just trying to help…”
    Same thing when you find anything questionable too – “With your data, I get this, but my finer grained data says that, and it could be a problem. If you check using the extra data, the assumptions that look good with the old style data don;t actually hold, this could be a problem…”
    It just HAS to be done this way – we’ve already tried the “thieves and liars” line, but all they do is claim worlds best practice, and the non-technical people stop at that point. So doing it the other way means they need to find a reason to NOT use it, and the harder that is to do, the more fine-grained tech they try and use to avoid it, the more they look like they have something to hide. And we are guaranteed to be able to get raw data, not “homogenised” crap!
    Likely be a while yet before this goes forward, Jennifer is still organising various people/sites etc, but as I said, keep an eye out for it.

  10. Larry Ledwick says:

    Interesting commentary here – I think his discussion explains why the climate data is so screwed up, they have normalized deviance and think what they are doing is normal for everyone, and just whistle as they walk through the grave yard of horrific data manipulations and misuse of methods.

    https://danluu.com/wat/

  11. E.M.Smith says:

    @Larry L:

    Yes, interesting article. I’ve worked at a bunch of places with various of those kinds of “quirks”. At Amdahl, you could be fired for possession of alcohol on campus. That was “normal” in the IBM mainframe world. When I left there for Apple, the next week I was handed a beer at the Friday Beer Bash – it was “normal” there… Something broke, I had to go fix it. I was at my desk madly fixing and took a swig of my…my… MY GOD BEER!!! Oh, wait, it is fine to have a beer at my desk on Friday here… It was “normal”…

    There’ a LONG list of others… At Disney, they have a culture, STRONGLY enforced, that a “Cast Member” is never to cause a Guest an uncomfortable feeling. Because this is indoctrinated into all Cast Members, it is strongly frowned upon to tell someone they are screwing up the process and need to get their job done, or right… Even if they desperately need it. Consequently, folks do all sorts of things “not quite right” and nobody tells them… Their boss is supposed to somehow just know and do corrections.. Also, everyone is terribly worried about saying things that might make someone feel bad. Very Stepford Wives in a way…

    And on and on… As a contractor I was in dozens of companies. Every one of them has “weird stuff” based on some quirk of their upper layers… At a Government Teaching Hospital we had a guy who came in, in a suit, every day and sat at a small desk in the filling room. He did nothing at all. The “story” was that he’d made a pass at a girl once a few years ago, and when it didn’t go well and he was reprimanded he twigged a bit… Firing him would just put him on State Welfare / Disability so why not just let him keep collecting his paycheck? Same source… Nobody talked to him. He just sat quietly for 8 hours (lunch in the middle) and went home each day. That was “normal”…

  12. Bill in Oz says:

    Kneel you want to replicate the BOM ? Bugger ! That would be a waste of time as it would be quickly infected with the same pius ideology of climate warming.

    I read Jennifer Marohasy’s stuff . It’s good. No bull and no pius kneeling to the BOM.

  13. Geoff Sherrington says:

    Bill in Oz.
    When you see a BOM station history with an early start date, that can be because it was when rainfall observations started. There can be 30 years or more before temperature observations started. You really have to look at the station metadata or similar, one by one. For the 100+ ACORN-SAT stations, refer to the BOM Acorn sat station catalogue, easily found by searching. Geoff

  14. Bill in Oz says:

    Geoff, Here in Mt Barker in SA the BOM has RAW temperature & rainfall records that start in 1863. I used them in my own post about Mt Barker’s weather/climate back in December.. But there is a 37 year gap in the published temperature record from the 1880′ to 1923 I think. The rainfall record however has no such gap. Very curious ! I suspect that data is being held unpublished.

    SAs for the ACORN-Sat stuff.. There is no need for me to do it. Jennifer Marohessy ( on IPA & on WUWT ) JoNova, Kenskingdom and yourself, do a good job of putting the needed axe through it

  15. cdquarles says:

    About seasonality. Where I live, Winter is the most variable season. Summer is the least. The two transition seasons, Spring and Autumn, trend from more variability early to little late for spring; and autumn is the opposite. Diurnal ranges follow that! In winter, we may see a diurnal range from 0 to 50 Fahrenheit degrees. We see days with rising temps. We see days with falling temps. And, that said, temperatures are themselves a geometric average of the actual internal kinetic energy (and only that) of a defined sample of matter. In case of fluids, said defined sample is that within some kind of container. That does not have to correspond to actual temperatures in flowing fluids.

Anything to say?

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.