Looking at Africa, I had this odd “Anoamly Differences” Graph for Dijibouti. I had to wonder why? There’s a movement of older temperature anomalies down, and more recent temperature anomalies up. There just can’t be that many thermometers in Dijibouti, and I really doubt they have a strong Met Department going back to fiddle the 1960s data.
GHCN v3.3 vs v4 DJ Dijibouti Difference
The straight line segments are what it ought to look like if not much is changing. There’s a very slight overall warming, but then a BIG dip between about 1960-1972, then more flat, then a big rise after 1990 or so.
That “dip” is centered nicely in the “baseline period” used by GIStemp and Hadley.
Here’s the actual anomalies plot:
GHCN v3.3 vs v4 DJ Dijibouti Anomaly
Until you take that “tuck” in the past, it’s mostly just a flat scatter up to about 1995, for the black dots. Then after that, there’s only one high black dot. The big “rise” comes out of the red ones. So what’s different? Why are those changes in what ought to be at most a very few thermometers? Well, I ran a little report:
MariaDB [temps]> SELECT year,num3,mean3,mean4,num4,abrev FROM yrcastats WHERE abrev='DJ' ORDER BY year; +------+------+-------+-------+------+-------+ | year | num3 | mean3 | mean4 | num4 | abrev | +------+------+-------+-------+------+-------+ | 1951 | 12 | -0.25 | -0.19 | 12 | DJ | | 1952 | 12 | -0.47 | -0.41 | 12 | DJ | | 1953 | 12 | -0.46 | -0.40 | 12 | DJ | | 1954 | 12 | -0.42 | -0.36 | 12 | DJ | | 1955 | 12 | -0.54 | -0.48 | 12 | DJ | | 1956 | 12 | -0.69 | -0.63 | 12 | DJ | | 1957 | 12 | -0.91 | -0.85 | 12 | DJ | | 1958 | 12 | -0.17 | -0.11 | 12 | DJ | | 1959 | 12 | -0.13 | -0.06 | 12 | DJ | | 1960 | 12 | -0.25 | -0.19 | 12 | DJ | | 1961 | 12 | 0.39 | 0.24 | 12 | DJ | | 1962 | 12 | -0.15 | -0.19 | 12 | DJ | | 1963 | 12 | 0.21 | -0.10 | 12 | DJ | | 1964 | 12 | -0.05 | -0.36 | 12 | DJ | | 1965 | 12 | -0.77 | -0.87 | 12 | DJ | | 1966 | 12 | -0.11 | -0.48 | 12 | DJ | | 1967 | 12 | -0.17 | -0.53 | 12 | DJ | | 1968 | 12 | -0.21 | -0.53 | 12 | DJ | | 1969 | 12 | 0.30 | -0.06 | 12 | DJ | | 1970 | 12 | 0.28 | -0.02 | 12 | DJ | | 1971 | 12 | -0.31 | -0.25 | 12 | DJ | | 1972 | 12 | 0.03 | 0.09 | 12 | DJ | | 1973 | 12 | 0.25 | 0.31 | 12 | DJ | | 1974 | 12 | -0.13 | -0.06 | 12 | DJ | | 1975 | 12 | 0.51 | 0.57 | 12 | DJ | | 1976 | 12 | 0.08 | 0.14 | 12 | DJ | | 1977 | 12 | 0.55 | 0.61 | 12 | DJ | | 1978 | 12 | 0.41 | 0.47 | 12 | DJ | | 1979 | 12 | -0.06 | 0.00 | 12 | DJ | | 1980 | 12 | 0.58 | 0.64 | 12 | DJ | | 1981 | 12 | 0.19 | 0.25 | 12 | DJ | | 1982 | 12 | 0.02 | 0.08 | 12 | DJ | | 1983 | 12 | 0.18 | 0.24 | 12 | DJ | | 1984 | 12 | -0.82 | -0.76 | 12 | DJ | | 1985 | 12 | 0.04 | 0.10 | 12 | DJ | | 1986 | 12 | 0.10 | 0.16 | 12 | DJ | | 1987 | 12 | 0.18 | 0.24 | 12 | DJ | | 1988 | 12 | 0.76 | 0.82 | 12 | DJ | | 1989 | 12 | 0.13 | 0.19 | 12 | DJ | | 1990 | 12 | 0.65 | 0.71 | 12 | DJ | | 1991 | 4 | -0.14 | -0.11 | 4 | DJ | | 1992 | 3 | -0.17 | -0.06 | 3 | DJ | | 1993 | 7 | -0.36 | -0.39 | 6 | DJ | | 1994 | 7 | -0.23 | -0.20 | 7 | DJ | | 1995 | 7 | 0.15 | 0.23 | 7 | DJ | | 1996 | 6 | 0.45 | 0.86 | 5 | DJ | | 1997 | 7 | 0.56 | 0.77 | 8 | DJ | | 1998 | 8 | 1.27 | 1.64 | 7 | DJ | | 1999 | 7 | 0.13 | 0.96 | 5 | DJ | +------+------+-------+-------+------+-------+ 49 rows in set (0.08 sec)
That “num3′ and “num4” is the number of monthly temperature data items in that YEAR. That’s one a month to make 12.
MariaDB [temps]> SELECT stnID,cnum FROM invent3 where cnum=114; +-------------+------+ | stnID | cnum | +-------------+------+ | 11463125000 | 114 | +-------------+------+ 1 row in set (1.70 sec) MariaDB [temps]> SELECT stnID,abrev FROM invent4 WHERE abrev='DJ'; +-------------+-------+ | stnID | abrev | +-------------+-------+ | DJM00063125 | DJ | +-------------+-------+ 1 row in set (4.58 sec)
There’s only one damn thermometer in the whole country.
All those early years have 12 records. What on earth would justify changing historical data from the same instrument? Every Single Year is DIFFERENT between v3.3 and v4. This is just a flat out Data Diddle.
In the recent end, when temperatures go a bit wild, we also lose about 1/2 the monthly records. Anyone want to bet they kept the ones in warm months?
I suspect this is the kind of thing we’ll see in lots of other countries too. “pruning” of what records are in the recent data to those with a warming anomaly to “shape the end” while putting the colder ones back in after a dozen years to “cool the past” when that is needed. Just speculation, but informed by the shape of the data and the reports.
I note in passing that the data end in 1999. That means the last 20 years will be infilled with fiction in the “homogenizing”. That, then, will likely splice on some OTHER stations rising tail “now” to this rise in the late 90s, and make a combined nice continuous “trend”.
If there’s some particular bit of information from the database anyone wants about these particular Stations, let me know and I’ll add it in a comment or update.
Musings from the Chiefio 
Taking a sample country and examining ( digging here) the data in detail is a great way of assessing the scientific validity of the data.
And yes, EM, it all seems shonky…
Meanwhile Djibouti has an interesting foreign presence in recent decades : American, French Chinese, Japanese & Italian. From Wikipedia:
“Djibouti’s strategic location by the Bab-el-Mandeb Strait, which separates the Gulf of Aden from the Red Sea and controls the approaches to the Suez Canal, has made it a desirable location for foreign military bases. Camp Lemonnier was abandoned by the French and later leased to the United States Central Command in 2001; the lease was renewed in 2014 for another 20 years.[61] The 13th Demi-Brigade of the French Foreign Legion is still stationed in Djibouti as the largest French military presence abroad, the only one commanded by a 3-star general. The country also hosts the only overseas Chinese support base and the only overseas Japanese military base.[62] The Italian National Support Military Base is also located in Djibouti.[63]”
Yet there have been NO temperature ecord kept since 1999 ?
Duhhhhh ?
That statement itself seems made up & made to create more dopiness. Why ?
Because every one of those military bases would have it’s own in house meteorological station.
I can’t see any of them relying on weather information from anyone else. ( Well maybe the Italians ! But China or Japan or France or the USA ? Nahhhhhh It does not happen
Soooooo. This to me indicates a suppression of weather data for the past 2 decades…After all the locals know what weather to expect seasonally..
How many people know where Djibouti is? So a little fiddle there helps push up the claimed temperature, much like those 2 towns Columbia with temperatures above 80℃ for 2 months (in the HADCRUT version). Or that station in east Africa that recorded ONE temperature for 40+ years.
Nero fiddling, not zero fiddling.
I think I’m going to add another field to the database. Number of thermometers in a given year for a given country…. the “num3” and “num4” fields are for calculating % data present vs missing “someday”, so are inflated by the 12 months in a year (compared to instrument count).
Then find the places without a lot of instrument changes to cover the diddle and see what changed…
@Bill in Oz:
Golly, I had no idea it was so heavy with military bases. Yeah, they will know EXACTLY what the temperature has been for the last 20 years…
@Graeme No3:
That does seem to be the overall game plan. No One Method so it doesn’t show up in the bulk statistics. Lots of small different diddles all over the place to make a ‘trend’. Some via homogenizing (likely for this station in the last 20 years – so looking at the anomaly graphs for stations ‘near it’ will be interesting), some via month dropping, some via station dropping some via direct bit-o-diddle, etc. No one bank robbery, just a lot of petty theft and pick pockets nobody reports…
Then mix that with a decade or two roll-out of the digital MMTS which sure looks to have a bias against recording low going intervals (either due to the instrument itself, or the location being tethered to a building / at airports, or perhaps the “splice / calibration” to the earlier instrument records)…
And suddenly you got yourself a passel of Global Warming Trend.
EM, I know there is data after 1999, my son a meteorologist was stationed there around 15 or 16. Whether it is reported to NOAA, dunno. BTW he was all over Africa installing weather stations for our and allies’ military. He also spent some time floating ARGO-style buoys on other tours.
Now what are the odds of this. Say you wanted to “Diddle the Data” but have your net change be nearly zero. A tuck down in the past, some bounce up i the present, don’t touch the last temperature as folks might notice, then gently tweak all the other temps just s smidge to about zero, right? Now someone adding up all the changes gets no net change. The statistics will look like “as much up as down”. Just the timing happens to land the cold dip in the “baseline” and the pop up in the now….
So I looked for that. Take the mean anomaly for all years in version 4, subtract the mean anomaly for all years for version 3.3, for all but the last year of data:
4/100 C is ALL the variance.
I think that is no accident. You don’t “accidentally” get that much change averaging to 0.0 C of change.
Another interesting statistical explore. I did some gross comparison of overall change. Overall, the record in total gets cooled. But the past gets cooled A LOT while the present, not so much. There’s a firm “tilt” to the cooling changes in the historical data:
In 2014 (the last year there is v3.3 data to compare) the data is slightly warmed in v4:
So an average of 1.38 of warming.
Divide by number of countries to get degrees / country. The yrcastats file is the average anomaly for each country in each year. There are 240 countries. So this year is warmed by 1.38/240 C in v4 vs v3.3 or 0.00575 C for this year.
Look at the last 3 years data, cooling starts to set in, in the past.
Take it out to the last 13 years of data, even cooler:
Then, look at everything prior to 1990 and the past is getting quite a lot colder!
So the past prior to 1990 is cooled by 8.3 C overall. But spread out over a lot of years.
IF we look at only the “baseline interval” it is much less:
“Only” 2.96 C, spread over about 40 years, or 0.074 C / year.
Realize this is NOT a rigorous statistical analysis. Different countries have different aggregations of thermometers, so ought to have different weights. I really need to do this “per instrument” for the deg C to mean anything.
What it DOES show is a consistent bias to “cool the past” in aggregate in GHCN v4 vs v3.3 and to do so with some vigor.
Quite something when you remember that those “num3>0 AND num4>0” screens are to assure we are ONLY comparing data for years with matching data. This is a “like for like” inside the two data sets. The “new” data in v4 is excluded as there is no matching v3.3 data (num3=0 in those years).
So by definition this is “cooling the history”.
The present database was designed for comparing countries, so it will take a bit of a think to see how best to compare other units (areas, thermometers, latitudes, etc.). But I think this is good enough to demonstrate a significant “Diddle” toward cooling the past in v4.
@EM: Since there is just the 1 instrument, can you do a run of the hi, low and avg for the 2 sets, and put them side-by side in a single report table with the low diff, hi diff and avg diff, i.e.,
date, v3low, v4low, lowdiff, v3hi, v4hi, hidiff, v3avg, v4 avg, avgdiff
Perhaps pick 1 year only. I’m interested to see if the diddle is on the nighttime, daytime or combination of the two.
I think so. How’s this?
Note I subtract the v4 from the v3 so “warming” will show up as a negative value and cooling as a positive one. This is backwards from the graphing program but keeps the fields in the same order in the print and the math. If you want things in some other order, let me know. It’s a one line program after all ;-)
It is wonderfully sad that stalwarts like our Chief and Tony Heller and the good people over at wuwt are the only ones to call out the crooks and shysters who are cooking the temperature books.
If only Orwell were alive to witness how right he was. 1984 is being acted out right before our eyes.
Instead of being at war with Eastasia we are in a constant propaganda war of lies.
Thank you Chiefo.
I’ll repost this at Jennifer Marohasy in Oz. She is on to temp data adjusting.
Considering this countries location, I am surprised all the thermometers had not been pirated away as booty. Just sayin……
@EM: Cool! Though, I had imagined the math with 4-3, rather than the 3-4.
This is more revealing than I had hoped. The outlier years (where the range changes dramatically) like 1961 and 1970 really do stand out. Also, the very constant average diff in the first few years appears to be the result of a series of exactly ofsetting changes in low and high.
It provokes my curiosity how these patterns could result from two different date series summaries derived from the same (supposed) series of measurements.
This is great stuff.
@EM: 1963 looks curious, too. How could the high average be different by 1.24 degrees?
Is it possible to do a day-by day for that year, perhaps to determine the specific ‘diddly-days’?
I have always suspected that this is how they were cooking the books lots of small tweaks in out of the way stations unlikely to be noticed by average people, and using poor data quality as an excuse to allow lots of undocumented adjustments. Just like an embezzler moving funds around to muddy the audit trail making it very hard to figure out who exactly did what.
The 1961-1990 WWR source data for DJIBOUTI is located here. Other years use different sources.
https://www.ncdc.noaa.gov/wdcmet/data-access-search-viewer-tools/world-weather-records-wwr-clearinghouse
DJIBOUTI in series 6 1961-1970 uses wmo#63126 data.
Series 7 1971-1980 uses wmo#63125 and also includes 1961-1970 data. Some other records in series 7 also have 1961-1980 data.
So, they are using two different datasets 1961-1970 with v3 getting all its data from 63125 and v4 using 63126 for 1961-1970 and then using 63125 for the rest of the years. That is why 1961-1970 is different.
If the data series are only supposed to be one decade long it looks like they screwed up building the WWR database and also use different coding to extract that data into the GHCN v3 and v4 databases.
Off Topic as it is not Djibouti. Raher it’s Australia with a complete summuray of the BOM’s ACORN 1 Vs ACORN 2..
It’s all a big hash !
It is long past time to hold those who are erasing the past accountable. The climate consensus has in effect become historical thieves stealing the past in an attempt to control the future.
@Steven Fraser:
OK, here it is with the fields in the other order. 4 then 3.
MariaDB [temps]> SELECT year,small4,small3,(small4-small3) -> AS 'Ldiff',big4,big3,(big4-big3) AS 'Bdiff', -> mean4,mean3,(mean4-mean3) AS 'Avdiff' FROM yrcastats -> WHERE abrev='DJ'; +------+--------+--------+-------+------+------+-------+-------+-------+--------+ | year | small4 | small3 | Ldiff | big4 | big3 | Bdiff | mean4 | mean3 | Avdiff | +------+--------+--------+-------+------+------+-------+-------+-------+--------+ | 1951 | -1.61 | -1.87 | 0.26 | 0.84 | 0.89 | -0.05 | -0.19 | -0.25 | 0.06 | | 1952 | -1.61 | -1.82 | 0.21 | 0.24 | 0.26 | -0.02 | -0.41 | -0.47 | 0.06 | | 1953 | -1.69 | -1.92 | 0.23 | 0.44 | 0.49 | -0.05 | -0.40 | -0.46 | 0.06 | | 1954 | -2.71 | -2.97 | 0.26 | 0.51 | 0.49 | 0.02 | -0.36 | -0.42 | 0.06 | | 1955 | -2.19 | -2.42 | 0.23 | 0.47 | 0.53 | -0.06 | -0.48 | -0.54 | 0.06 | | 1956 | -3.21 | -3.47 | 0.26 | 0.24 | 0.26 | -0.02 | -0.63 | -0.69 | 0.06 | | 1957 | -2.11 | -2.37 | 0.26 | 0.44 | 0.49 | -0.05 | -0.85 | -0.91 | 0.06 | | 1958 | -1.39 | -1.62 | 0.23 | 0.84 | 0.86 | -0.02 | -0.11 | -0.17 | 0.06 | | 1959 | -1.51 | -1.77 | 0.26 | 1.04 | 1.06 | -0.02 | -0.06 | -0.13 | 0.07 | | 1960 | -1.49 | -1.72 | 0.23 | 0.39 | 0.43 | -0.04 | -0.19 | -0.25 | 0.06 | | 1961 | -1.09 | -0.02 | -1.07 | 0.74 | 1.27 | -0.53 | 0.24 | 0.39 | -0.15 | | 1962 | -1.39 | -1.44 | 0.05 | 0.71 | 1.48 | -0.77 | -0.19 | -0.15 | -0.04 | | 1963 | -0.99 | -0.62 | -0.37 | 0.79 | 2.03 | -1.24 | -0.10 | 0.21 | -0.31 | | 1964 | -1.19 | -1.07 | -0.12 | 0.51 | 0.87 | -0.36 | -0.36 | -0.05 | -0.31 | | 1965 | -1.59 | -1.51 | -0.08 | 0.34 | 0.13 | 0.21 | -0.87 | -0.77 | -0.10 | | 1966 | -1.09 | -1.07 | -0.02 | 0.14 | 0.88 | -0.74 | -0.48 | -0.11 | -0.37 | | 1967 | -1.19 | -0.94 | -0.25 | 0.39 | 1.07 | -0.68 | -0.53 | -0.17 | -0.36 | | 1968 | -1.52 | -1.23 | -0.29 | 0.47 | 0.78 | -0.31 | -0.53 | -0.21 | -0.32 | | 1969 | -0.89 | -0.64 | -0.25 | 1.21 | 1.56 | -0.35 | -0.06 | 0.30 | -0.36 | | 1970 | -0.59 | -1.11 | 0.52 | 0.58 | 1.17 | -0.59 | -0.02 | 0.28 | -0.30 | | 1971 | -1.39 | -1.41 | 0.02 | 0.61 | 0.48 | 0.13 | -0.25 | -0.31 | 0.06 | | 1972 | -0.56 | -0.63 | 0.07 | 0.91 | 0.83 | 0.08 | 0.09 | 0.03 | 0.06 | | 1973 | -0.73 | -0.67 | -0.06 | 1.71 | 1.48 | 0.23 | 0.31 | 0.25 | 0.06 | | 1974 | -1.06 | -1.01 | -0.05 | 1.31 | 1.08 | 0.23 | -0.06 | -0.13 | 0.07 | | 1975 | -0.19 | -0.14 | -0.05 | 2.01 | 1.88 | 0.13 | 0.57 | 0.51 | 0.06 | | 1976 | -0.36 | -0.33 | -0.03 | 1.01 | 0.79 | 0.22 | 0.14 | 0.08 | 0.06 | | 1977 | -0.89 | -0.91 | 0.02 | 2.61 | 2.38 | 0.23 | 0.61 | 0.55 | 0.06 | | 1978 | -0.29 | -0.52 | 0.23 | 1.61 | 1.48 | 0.13 | 0.47 | 0.41 | 0.06 | | 1979 | -0.73 | -0.67 | -0.06 | 0.61 | 0.38 | 0.23 | 0.00 | -0.06 | 0.06 | | 1980 | -0.33 | -0.33 | 0.00 | 2.31 | 2.18 | 0.13 | 0.64 | 0.58 | 0.06 | | 1981 | -0.43 | -0.42 | -0.01 | 1.09 | 0.83 | 0.26 | 0.25 | 0.19 | 0.06 | | 1982 | -1.26 | -1.24 | -0.02 | 1.49 | 1.23 | 0.26 | 0.08 | 0.02 | 0.06 | | 1983 | -0.76 | -0.72 | -0.04 | 2.29 | 2.03 | 0.26 | 0.24 | 0.18 | 0.06 | | 1984 | -1.89 | -1.91 | 0.02 | 0.81 | 0.68 | 0.13 | -0.76 | -0.82 | 0.06 | | 1985 | -0.83 | -0.82 | -0.01 | 1.99 | 1.73 | 0.26 | 0.10 | 0.04 | 0.06 | | 1986 | -1.39 | -1.34 | -0.05 | 2.41 | 2.28 | 0.13 | 0.16 | 0.10 | 0.06 | | 1987 | -1.39 | -1.62 | 0.23 | 1.99 | 1.73 | 0.26 | 0.24 | 0.18 | 0.06 | | 1988 | -0.56 | -0.51 | -0.05 | 2.09 | 1.97 | 0.12 | 0.82 | 0.76 | 0.06 | | 1989 | -1.26 | -1.33 | 0.07 | 0.79 | 0.73 | 0.06 | 0.19 | 0.13 | 0.06 | | 1990 | -0.23 | -0.17 | -0.06 | 1.81 | 1.79 | 0.02 | 0.71 | 0.65 | 0.06 | | 1991 | -1.11 | -1.23 | 0.12 | 0.71 | 0.76 | -0.05 | -0.11 | -0.14 | 0.03 | | 1992 | -0.41 | -0.53 | 0.12 | 0.34 | 0.39 | -0.05 | -0.06 | -0.17 | 0.11 | | 1993 | -1.71 | -1.97 | 0.26 | 0.61 | 0.66 | -0.05 | -0.39 | -0.36 | -0.03 | | 1994 | -0.96 | -0.94 | -0.02 | 0.61 | 0.38 | 0.23 | -0.20 | -0.23 | 0.03 | | 1995 | -1.09 | -1.22 | 0.13 | 0.87 | 0.93 | -0.06 | 0.23 | 0.15 | 0.08 | | 1996 | 0.41 | -0.07 | 0.48 | 1.44 | 1.06 | 0.38 | 0.86 | 0.45 | 0.41 | | 1997 | -0.11 | -0.04 | -0.07 | 2.46 | 1.48 | 0.98 | 0.77 | 0.56 | 0.21 | | 1998 | 0.27 | 0.43 | -0.16 | 2.13 | 1.79 | 0.34 | 1.64 | 1.27 | 0.37 | | 1999 | 0.61 | -0.72 | 1.33 | 1.46 | 0.56 | 0.90 | 0.96 | 0.13 | 0.83 | +------+--------+--------+-------+------+------+-------+-------+-------+--------+ 49 rows in set (0.79 sec) MariaDB [temps]>Having spent a significant time in Djibouti it does have me scratch my head as to why no data. Far more thermometers now than in 1999 or before. I wouldn’t be surprised if there were a decrease in temperatures due to the increase in vegetation. Still hot hot hot and humid but the increased humidity from irrigation probably has a negative effect on daily highs.
Don’t go there unless you have an up to date shot record and carry it with you…
@Patrick Healy:
You are most welcome.
@Steven Fraser:
I don’t have daily data for Djibouti. (As far as I know). I think I grabbed one daily series in my “trawl” but I think that was just the USHCN. The GHCN is claimed to be monthly averages only (or at least was when I first dug into it a decade ago….) and that the upstream countries might have processed the “unadjusted” data in some way during the averaging so don’t expect “unadmusted” to mean “raw”.
The short answer is: Maybe with a month or two of work… I’d be starting from scratch on any daily data based operations. I’m sorry, but with only one of me I have to “focus”, and for now I’m focused on GHCN v3.3 vs v4 (and that is monthly data) with Africa the in process group right now. On Deck is to add v1 and v2 to the comparison, then look at Min Max data from the same sets. Probably at least a year for me to “Get it done”. (Now if a nice Money Bags wants to toss $1 Million at me to hire a couple of staff I can likely get it done in months or weeks… but don’t plan on it. There isn’t much money available on this side of things…). Waaaay down the line is “do daily data”.
Why? Time & money & processing power. Daily Data will take about 30 x as much storage and computes. As this pretty much loads up a R. Pi M3 for 10 minute chunks and is about 8 GB, you are talking about an 8 x 30 = 240 GB database, a need for about 2 x 30 = 60 GB of memory (or 2 GB with less in parallel) and about 10 x 30 = 300 minute, or 5 hour runs PER STEP. Either that, OR buying a bigger hotter machine to do it with. I don’t have the money for a 30 x faster machine… (I MIGHT be able to do it with a bunch of R.Pi running in parallel as there is a parallel DB ending for MySQL / Maria DB, but I have no experience with it nor is it likely ported to the Pi just yet.
So with just one of me, the hardware my $Few Hundred of donations buys, and no big funding, this is what I can do at the pace I can do it.
@Bob Koss:
Good stuff! I’m unfamiliar with the “world weather records clearinghouse”. I’ll need to look into that. I wonder if it came about after our (skeptics) 2 decades of complaints about no decent records kept…
@Bill in Oz:
I’ll take a look at that in a bit more depth, but later, and only after I’ve got this series of graphs wrapped up. I really really want to get this in the rear view mirror! (Making v3.3 vs v4 graphs…). It’s rather tedious… and I’m not (yet) seeing a decent way to automate it any further.
Looks like another interesting approach is to look at the average of all the changes for a country.
For South Africa, the past is all cooled about 1/4 C to 1/3 C up until the “baseline period” when it warms some, then there’s a “dip” tossed in for changes in recent data, then a bit of rise at the end. You would think that with all those changes (and in multiple instruments for a country as large as South Africa) there would be some kind of “net change”.
But Noooo….. The AVERAGE of ALL changes between v3.3 and v4 just accidentally end up as a (truncated) 0.0 C for South Africa. 9/100 C if you round instead.
Now with basically Every Year having changes, and some of them quite large ( 1/2 to 3/4 C range post 2000), you would think there would be more variation than that in the average of them all.
To get that little average change in the whole country, from things like correcting errors or removing a bogus value or two, while having every year change, in my opinion requires a computer tailored “fit” of the changes to the historical averages…
So looks to me like “Yet Another Diddle Discovery Tool”… So sum over all but last year for any one instrument (or record made by merging into one record), and average of all changes over all instruments for a country. IMHO reverse engineer that and you get the Diddle Process…
I know GHCNM been using WWR data as one of their sources since at least 2011.
E M : Re The ACORN 1 & 2 from BOM. Yes I know you are still busy completing the charts for Africa and then there is Europe to go as well. Stay focussed. Your work on this is great. But when you get a chance looking at what Ken has done here in Oz may see a few light glove moments. Unfortunately he does not do a comparison with raw BOM data and ACORN 1 & 2.
I suspect that would be even more revealing about BOM diddling of the figures.
Just a small progress report:
ALL graphs for Africa have been made. That’s something like 62 countries and 124 graphs or reports that it isn’t in v3.3. All of them have been uploaded and are in the draft report.
With that, I’m taking a break until tomorrow from working on this. Next I get to look at all the pairs of graphs, and at the neighbor country set, and observes what there is to observe, and type a note. I think I can get that done tomorrow. At 2 minutes per evaluation, figure about 240 minutes, or 4 hours. With coffee….
The “good news” is Africa is the biggest one. Europe, while also big, is smaller in number of countries… So I’m over the hump on the total countries (by a lot) and over the hump on “most countries in a report / posting.
So my fingers are tired and hurt ( a fair amount of typing involved in doing all this…) and my neck is a bit stiff (staring in the same direction at the same screen for hours…) so time for some “Qality Mutt Time” with the dogs in my lap and “mind-rot” on TV ;-)
@EM: your efforts are very much appreciated. Thanks for the alternately-ordered run of the program.
I’m going to poke around to see if the ‘raw’ daily data are available anywhere. might even be out there in newspaper archives, which would be fun to compare with the ‘recorded’ values, even in a limited scope.
And, if I happen to see that $1M on the sidewalk, I may look you up…
@EM: Dinner renewed my energy, so I went back and poured the most recent chart into an Excel spreadsheet, so I could do some ‘eyeballing’ of the whole result set, and look for patterns.
As your graph shows, there are 4 sections to this report, 2 where the change between v3 and v4 are very consistent, and graph horizontally as a line of dots, and 2 sections with ‘variance’ of distribution. For lack of a better term, I think of those as ‘regimes’, with these date ranges:
1951-1960 Regime #1 – 10 years, steady
1961-1970 Regime #2 – 10 years, variant
1971-1990 Regime #3 – 20 years, steady
1991-1999 Regime #4 – 9 years, variant
In the ‘steady’ decades, 30 of the Avdiff values are 0.06, and 2 are 0.07. IMO, probably a rounding effect somewhere.
In the ‘variant’ decade, and the final 9 years, all the Avdiff values are all negative in the decade, varying from -.1 to -.37, with an average value of -.262 in 1961-1970, and varying to the positive in the last 9 years, ranging from -.03 to 0.41, average 0.22667. There is 1 and only 1 value which breaks this pattern, the -0.03 in 1993.
In my opinion, these patterns of values could not occur in nature. 3 decades of Avdiffa values of 0.06 is highly unlikely. On top of that, at two decadal boundaries, the regime changes from the steady to variant patterns, and then back. Highly curious for me.
The next thing I looked at was the year-to-year differences between the Ldiff and Bdiff columns.
This analysis revealed the stability in the ‘steady’ years, and the turbulence in the
‘variant’ years. It also reveals that the steadyness of the distribution in 2-decade period occurred differently.
In the first decade, the sum of the Ldiff year-to-year values is -0.03, and the sum of the Bdiff year-to-year values is 0.01. These values almost completely compensate for each other.
In the second decade, the year-to-year variations in Ldiff and Bdiff diverge. The sum of the Ldiff year-to-year difference values are 1.59, while those of the Bdiff series sum to -.06. The disparity which accounts for the higher Avdiff values in this time period is due almost entirely to higher Ldiff numbers.
In the 20-year period, things appear differently, returning to year-to-year sums of -.08 for Ldiff, and -.11 for Bdiff. In this period, the values in the Ldiff and Bdiff columns have a wider range (Ldiff: -0.29 to 0.28; Bdiff: -0.14 to 0.15) but the values by-and-large compensate for each other over the course of the 2 decades, arriving at the 0.06 Avdiff value nonetheless.
Of additional interest is the period from 1982 to 1986, where the downward Bdiff year-to-year values were exactly compensated for by the subsequent year: Values: 0,0; -0.13, 0.13; -0.13, 0.13.
In the final 9-year period, THe Liff and Bdiff year-to-year numbers diverge again, with the Ldiff sum of 1.21, the Bdiff sum at 0.95, much like the pattern of the 2nd (variant) decade, but with higher values for the Bdiff year-to-year. In this period, both the Ldiff and the Bdiff values grew.
Verrrrry Interesting.
In summary, the ‘anthropogenic’ element I can detect in the numbers is a decadally-aligned series of patterns in the numbers, in which the Avdiff shift characteristics are achieved in a variety of ways.
Yeah, “Gupta’s Law of Creative Anomalies” applies… if it looks too good to be true, it isn’t. (h/t to the movie ‘Tomorrow Never Dies’).
@ALL: Pardon the typo. In the steady decades, 28 (not 30) of the Avdiff values were 0.06.
Pingback: GHCN v3.3 vs. v4 Anomaly Graphs – Africa | Musings from the Chiefio
Go to Wolfram Alpha, in the search bar type in: DIJIBOUTI, TEMPERATURE
You should get this…
https://www.wolframalpha.com/input/?i=Dijibouti,+TEMPERATURE
NOTE – It’s for this actual location, close to Dijibouti.
https://www4f.wolframalpha.com/Calculate/MSP/MSP933210ccd89i7ad4ad490000276f3604893i95i7?MSPStoreType=image/gif&s=43
Click on the drop down box where you read “current week” and select “all”.
The temperature profile looks NOTHING like the NASA GISS distortion of reality. I’ll be interested in any responses.
PS – Here’s the website of the location for the temps referenced in my WA link, above.
http://avia-pro.net/blog/aeroport-dzhibuti-ambuli
So, that thermometer has a coastal location. Not sure what the temps are doing inland, but even though they are probably hotter, it’s hard to imagine them going up, while on the coast they are going down.
@Yonason-Thanks for that.
I like walnut and chocolate fudge, but I’m getting sick of NASA-GISS fudge. Uuuuurpp! ;o)
Good show. Thanks again.
LOL
Your welcome, H.R.
I’ve searched other cities on W.A., and few agree with NASA GISS. And the funny thing is that many are at airports. So, if what should be the hottest data are showing the adjusters to be wrong, shame on the adjusters!
Pingback: GHCN v3.3 vs v4 – Top Level Entry Point | Musings from the Chiefio