What Is “The Smith Effect”
I’ve decided to start calling that mysterious property of the data that lets location bias leak through an analysis “The Smith Effect”. I first saw a great example of it in Canada and the investigation into Canada rapidly lead to Fort Smith. (But I’ll leave it for others to decide if The Smith Effect is named for Fort Smith or “some other Smith” ;-)
While running my experimental version of dT/dt that computes dT/year going forward in time, but computes “accumulated dT” going backwards in time, I’ve seen a fairly consistent pattern. Thermometer change still shows through. (BTW, I will start using rdT/dt as short hand for this reversed time dT/dt).
Exactly ‘why’ will be left an open question for a little while. Part of it, I’m fairly sure, is the way the data have been changed in their processing. We have an example of the TOBS adjustment bias in USHCN. While the GHCN ought not to have that particular bias in it, there is still the possibility that something in the Data QA process is introducing bias.
USHCN Adjustments With Rises and Steps
Because when you look at this data, you get a very clear ‘step function’ around the time of significant thermometer count changes. In “standard anomaly processing theory” that can’t happen. But in the real world it does. The bulk of the reason, IMHO, is The Smith Effect. I believe it is mostly the “location bias” of the thermometers that survive.
I have a working theory as to what The Smith Effect is, and what causes it. I now have a tool that surfaces examples of it fairly reliably. The tool is based on anomaly processing of a sort ( while rdT/dt does not use a baseline period, it does use year-to-year differences to create a history of anomalies rather than using temperatures directly ). So I am now in a position to build my case that my theory about the causal agent for The Smith Effect is in fact the valid cause.
I will not be discussing what the theory is behind The Smith Effect. At least, not until I’ve found out if it’s going into a paper for publication, or not. Once that is resolved, I’ll either post the details here (if it’s not going to publication) or post a pointer to the publication (so we all can sit on pins and needles waiting ;-)
With that said, lets take a look at Japan.
This first graph is the “Hair Graph” for Japan. Here we see the tendency to a ‘step function’ at thermometer count changes. Japan has a more or less overall flat character, but that depends a fair degree on the drop at ‘the start of time’ when there are just a few thermometers (starting at one). As in all the other cases of a ‘hockey stick at the end” we have a compression of volatility of the monthly extremes at The Splice between different series. The net result is a warming trend. But is it real?
Japan dT by Month and cumulative change of temperature
If we divide the history into segments based on reasonable criteria, we see a very different picture. The first segment is the “single digit” segment. While it has a strong cooling trend, that is likely an artifact of the 90% change of thermometer counts with the edge effects of temperature series ending. We start with one thermometer and grow to 13.
The next segment is the ‘low double digit’ segment. Things have basically stabilized and the trend is essentially flat. We start with a single year jump to 22 thermometers, then rise steadily to 131. In my opinion, the stability of this record through so much thermometer change is a striking testimony to the care used in those years.
The third segment is the “post WWII” segment. We have a step function up (perhaps due to the explosion of thermometer counts, or perhaps do to procedural changes under the U.S. influence in post war Japan). Again, that segment has a flat trend. We start with 143 and end with 151. While we are ‘running hotter’ than the prior segment as a step function higher, the trend is essentially nil.
The final segment is after The Great Dying of Thermometers. We plummet to 50 thermometers, and gain a rapid rising trend. To me this speaks volumes. The recent “warming” in Japan has nothing to do with CO2, and everything to do with instrument change (or, more likely, change of the data after collection via “adjustments”).
Japan monthly dT and cumulative temperature change by Count segment
Japan, Land of the Rising Sun
My bias: I have a great admiration of Japan and the Japanese culture. I’ve spent many years practicing Shotokan Karate (and dabbled in one or two other martial arts). I’ve worked for a company with significant Japanese ownership. I’ve managed a facility (albeit mostly remotely) in Japan for a couple of years. I’ve been to Japan, and loved it. There is an attention to detail, precision, and accuracy that is profound. There is a moral sense of honor rivaled only by the Knights of Europe (be they English or Templar or…) and a few other such groups.
These are a people who “get it right” far more than they “get it wrong” when performing work tasks. (Yes, they do make errors from time to time, as do we all. But the consequence of social disgrace and shame is far far higher, and the attempt to avoid such consequence can be extreme). So, IMHO, there is little chance that the folks in Japan reading these thermometers are doing it haphazardly or with lack of diligence. It will be seen as A Duty, and every Duty is to be performed with as much skill as can be applied, every single time. While talking one’s own life for “failure to perform a duty well” has fallen from common usage, it still does happen from time to time and the idea of it still carries through the culture. So my bias it to believe that if there is something odd in the data, it is NOT due to the folks collecting the data in Japan. It will come from the selection bias of the folks at NCDC as they filter, adjust and select the stations that make it into the GHCN.
OK, that is my bias. What do the data say? In the following report, notice that dT column. That is the accumulated “change of temperature” as we move back through time. It is darned near flat (with about 1/2 C of “jitter” or “roll”) until about 1989-1990 when the thermometer counts change. We then get what is substantially a ‘step function’ to about -1.5 C of offset, though still with some year to year ‘jitter’ and some hints of multi-year rolling.
BTW, to the inevitable claim that this might be the program with a bug: You also see this pattern in the raw data as a simple average; though not as clearly. It is in the data, not in the program. With that said, I do need to point out that there are some interesting shifts of exactly what year the step is in, between the two programs. There is the potential for an “off by one” type of difference as things are calculated and accumulated into different years depending on the direction of time… This is complicated by the fact that the raw temperatures program just averages what it’s got for a year, while the dT/dt process does it by month and caries forward data through missing months. In the year of thermometer drops, that happens mid year, so some months will have ‘carry forward’ while others will not.
210 JAPAN
-rw-rw-r-- 1 chiefio chiefio 2122537 Mar 1 11:05 ./DTemps/Temps.rM210
Doing dT/dt Yearlies: lrmdyears
Produced from input file: ./DTemps/Temps.rM210
Thermometer Records, Average of Monthly dT/dt, Yearly running total
by Year Across Month, with a count of thermometer records in that year
-----------------------------------------------------------------------------------
YEAR dT dT/yr Count JAN FEB MAR APR MAY JUN JULY AUG SEPT OCT NOV DEC
-----------------------------------------------------------------------------------
2010 0.02 -0.02 46 -0.2 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
2009 -0.04 0.06 47 0.4 2.5 -0.3 -0.1 0.5 0.6 -1.2 -0.3 -0.6 -0.5 0.4 -0.7
2008 0.26 -0.30 49 -1.0 -2.5 0.4 0.8 -0.1 -1.0 1.5 -1.0 -0.8 0.0 0.0 0.1
2007 -0.11 0.37 56 1.5 1.3 0.6 0.4 0.0 0.6 -0.6 0.0 1.5 0.0 -0.9 0.0
2006 -0.28 0.17 53 -0.3 0.7 0.4 -1.3 0.8 -1.0 0.0 0.3 -1.0 -0.1 0.6 3.0
2005 0.47 -0.75 53 0.0 -1.6 -0.9 -0.2 -1.6 0.0 -1.3 0.5 0.2 0.8 -1.3 -3.6
2004 -0.28 0.75 51 0.5 1.0 0.9 0.0 0.7 0.7 3.0 0.8 0.3 0.6 -0.1 0.6
2003 -0.04 -0.24 54 -1.3 -0.7 -2.3 -0.7 0.0 0.2 -2.2 -0.3 0.4 -0.4 3.4 1.0
2002 -0.29 0.25 54 1.5 1.2 1.8 1.0 0.0 -0.5 -0.5 0.0 0.3 -0.4 -1.6 0.2
2001 -0.03 -0.27 54 -1.8 0.8 0.2 0.7 0.0 0.3 0.0 -0.9 -0.9 -0.1 -0.7 -0.8
2000 0.17 -0.20 53 0.6 -0.9 -1.1 -0.4 0.0 -0.4 1.1 -0.1 -1.1 -0.2 0.0 0.1
1999 0.51 -0.33 54 0.6 -1.1 0.0 -2.0 -1.2 0.6 -0.3 0.4 0.6 -0.8 0.2 -1.0
1998 -0.08 0.59 54 -0.3 0.8 0.1 1.7 1.3 0.0 0.0 0.7 1.5 2.2 -0.9 0.0
1997 -0.63 0.54 58 0.1 0.9 1.1 2.2 1.1 0.0 0.0 0.0 0.0 -0.4 0.8 0.7
1996 -0.35 -0.28 58 0.0 -0.6 0.0 -2.1 -0.9 0.9 -0.5 -0.9 0.0 -0.7 0.7 0.8
1995 0.29 -0.64 50 0.0 -0.5 0.9 -0.6 -0.7 -0.9 -1.2 -0.6 -1.2 0.0 -1.5 -1.4
1994 -0.59 0.88 58 -0.9 -0.4 -0.5 1.3 1.1 0.6 3.4 2.7 1.5 1.5 0.0 0.3
1993 -0.31 -0.28 58 0.0 0.6 -1.0 -0.9 0.2 0.3 -1.2 -1.0 -0.3 -0.3 0.7 -0.5
1992 -0.24 -0.07 50 0.0 0.0 0.0 0.0 -0.1 -0.4 -0.1 0.0 -0.1 -0.1 0.0 0.0
1991 0.05 -0.29 50 0.6 -1.7 -0.1 0.4 0.0 0.1 -0.2 -1.0 -0.2 -0.2 -1.2 0.0
1990 -0.53 0.58 157 -2.5 1.0 0.6 -0.5 0.6 2.1 1.1 1.1 0.4 1.1 1.1 0.9
1989 -1.28 0.75 157 0.8 2.6 1.2 1.0 0.0 -1.1 0.8 0.0 0.4 0.1 2.6 0.6
1988 -0.78 -0.51 157 1.2 -1.4 -0.6 -0.1 -0.4 0.0 -1.6 0.2 0.4 -1.4 -2.0 -0.4
1987 -1.59 0.82 157 1.8 2.5 0.8 -0.2 0.5 0.5 1.5 -0.2 -0.5 2.4 1.0 -0.3
1986 -1.05 -0.54 157 0.2 -2.5 -0.9 -0.2 -1.0 0.4 -1.4 -1.0 -0.2 -1.6 -0.3 2.0
1985 -1.68 0.63 157 0.0 3.1 2.9 1.6 1.4 -1.6 -0.5 0.2 0.8 0.9 -0.3 -0.9
1984 -1.17 -0.51 157 -2.3 -1.8 -2.5 -3.2 -1.3 2.0 1.9 0.4 -0.7 -0.3 0.8 0.9
1983 -1.15 -0.02 157 0.8 -0.1 -0.9 2.4 -0.5 -0.4 0.4 0.8 1.4 -0.4 -1.9 -1.9
1982 -1.73 0.58 157 1.4 0.0 0.8 -0.2 2.1 0.0 -2.3 0.3 0.3 0.7 3.1 0.8
1981 -1.57 -0.17 157 -1.8 0.8 0.2 0.8 -1.1 -1.6 1.9 1.4 -0.5 -0.4 -2.4 0.7
1980 -0.59 -0.97 158 -1.0 -3.5 -0.4 -0.3 0.9 -0.2 -0.5 -2.2 -0.9 -1.2 0.0 -2.4
1979 -0.92 0.33 158 1.0 3.8 0.7 -0.1 -1.0 0.2 -2.5 -0.6 0.1 1.4 0.3 0.6
1978 -1.14 0.23 158 2.4 0.0 -0.9 -0.8 0.6 1.3 1.2 1.9 -0.4 -1.2 -1.2 -0.2
1977 -1.66 0.52 158 -1.7 -3.2 0.4 0.7 -0.1 0.0 1.7 0.2 2.1 1.4 2.9 1.8
1976 -1.13 -0.53 157 -0.1 2.3 0.5 -0.7 0.1 -0.3 -1.2 -1.2 -3.0 -0.8 -1.9 0.0
1975 -1.58 0.44 157 0.1 -0.4 0.4 0.1 -0.5 0.4 1.2 0.1 2.1 0.3 1.5 0.0
1974 -1.04 -0.53 157 -2.0 -1.8 -0.2 -1.2 0.5 0.3 -1.8 -1.0 -0.2 0.2 -0.1 0.9
1973 -0.90 -0.14 157 -0.5 0.8 -1.2 1.4 -0.1 -0.5 0.7 1.2 0.0 -0.5 -0.7 -2.3
1972 -1.38 0.48 156 2.3 0.7 1.2 0.2 0.3 0.0 -0.4 0.1 0.2 1.1 -0.2 0.3
1971 -1.48 0.10 157 0.8 -0.5 2.2 0.6 -1.0 0.7 0.4 -0.4 -1.5 -1.2 0.2 0.9
1970 -1.43 -0.05 157 -1.5 0.3 -1.9 -0.9 0.4 0.0 0.4 0.2 0.5 1.1 0.2 0.6
1969 -1.30 -0.13 157 0.5 2.3 -1.4 0.0 0.7 -0.5 0.0 0.2 0.8 0.3 -1.1 -3.4
1968 -1.13 -0.18 161 0.7 -1.7 0.0 -0.1 -1.7 -0.7 -0.9 -1.1 -0.5 -0.6 0.7 3.8
1967 -1.23 0.10 159 -0.5 -1.7 -0.7 0.4 1.7 1.3 1.2 0.3 0.3 -0.8 0.0 -0.3
1966 -1.75 0.53 159 -0.4 1.6 2.9 1.9 -0.2 -0.6 0.0 0.5 0.8 1.2 -0.5 -0.9
1965 -1.00 -0.75 157 -1.0 0.8 -1.3 -4.3 -0.9 0.3 -0.9 -0.7 -1.0 -0.4 0.9 -0.5
1964 -1.31 0.31 158 3.1 0.0 -0.1 1.6 -0.3 -1.0 0.0 0.8 0.9 0.2 -0.7 -0.8
1963 -1.10 -0.21 157 -1.7 -1.5 0.0 0.5 0.7 0.9 0.1 -0.4 -1.7 -0.2 0.4 0.4
1962 -0.49 -0.61 156 0.8 1.0 -0.7 -0.7 -0.7 -1.0 -1.2 -0.3 -1.3 -2.0 -1.2 0.0
1961 -0.85 0.36 155 -0.9 -2.1 -0.7 1.4 1.1 0.7 1.0 0.1 1.3 1.5 0.1 0.8
1960 -0.53 -0.32 156 0.3 -1.2 -0.1 -1.5 -0.4 0.2 0.0 0.5 -0.1 -0.5 0.0 -1.0
1959 -0.91 0.38 155 -0.7 1.6 1.2 0.5 0.6 -0.7 0.2 0.5 0.5 1.1 0.6 -0.9
1958 -1.42 0.51 155 -0.7 1.8 1.7 0.0 0.2 1.4 0.6 -0.5 2.0 -0.2 -0.8 0.6
1957 -1.43 0.01 154 1.2 0.0 -2.0 0.5 -0.5 -1.3 -0.2 1.1 -1.9 -0.8 1.3 2.7
1956 -0.79 -0.63 152 0.5 -1.9 -0.6 -0.6 0.2 -0.6 -1.6 -1.1 0.4 0.4 0.3 -3.0
1955 -1.17 0.37 151 -1.6 -0.1 1.1 -0.3 0.0 2.8 3.3 -0.1 -1.0 1.1 -1.4 0.7
1954 -1.38 0.22 151 1.6 1.8 -1.0 2.1 0.4 -1.6 -1.6 0.5 0.8 -1.3 1.6 -0.7
1953 -1.43 0.04 150 -0.8 0.6 1.4 -1.1 -0.4 0.0 0.1 -0.5 0.0 0.7 -1.7 2.2
1952 -1.38 -0.05 147 0.8 -1.5 -0.2 0.4 -0.1 0.4 0.4 -0.8 1.7 -0.7 1.1 -2.1
1951 -1.08 -0.30 147 -1.6 0.0 0.0 -0.8 -0.7 -0.5 -1.6 0.5 -2.0 1.3 -0.3 2.1
1950 -1.51 0.43 143 -0.3 -1.3 1.3 2.3 0.4 1.1 1.3 0.3 0.3 -0.1 0.8 -0.9
1949 -0.88 -0.62 143 0.8 1.4 -0.9 -2.5 0.0 -1.3 -1.0 0.0 -0.1 -1.1 -0.7 -2.1
1948 -2.24 1.36 132 0.2 2.4 1.1 1.6 1.6 1.8 0.2 -0.4 0.2 1.8 1.3 4.5
1947 -1.31 -0.93 131 0.3 -1.7 -0.2 -1.3 -0.3 -2.7 -0.5 0.1 0.1 -1.4 -2.7 -0.9
1946 -2.38 1.08 124 2.7 2.3 -0.4 0.6 1.1 1.7 3.1 0.1 0.0 0.1 1.5 0.1
1945 -1.91 -0.47 124 -1.9 -1.1 0.5 1.9 -2.2 -1.1 -2.7 -0.2 -0.2 0.4 0.0 0.9
1944 -1.71 -0.20 123 0.7 -0.4 -0.7 -0.4 0.6 0.4 0.3 -0.1 -0.8 -0.3 0.6 -2.3
1943 -1.55 -0.16 123 -0.4 0.4 -2.4 -1.1 0.5 0.0 -1.4 0.3 0.3 0.7 0.4 0.8
1942 -1.60 0.05 111 -1.9 -1.4 1.5 0.5 -0.8 0.3 3.1 1.2 1.9 -0.2 -1.9 -1.7
1941 -1.64 0.04 110 2.2 1.0 0.7 0.1 0.2 0.1 -2.1 0.0 -0.7 -0.8 -0.3 0.1
1940 -1.57 -0.08 107 0.1 -0.2 0.1 -0.3 0.2 0.0 -0.2 -0.8 -1.1 0.0 0.4 0.9
1939 -1.56 -0.01 100 -0.6 0.3 -1.5 -0.4 -1.3 0.0 1.0 0.0 0.9 0.0 1.4 0.1
1938 -1.36 -0.20 95 -1.5 -2.6 1.0 0.2 1.2 0.9 -0.9 -0.8 0.0 0.5 -1.1 0.7
1937 -2.08 0.72 91 2.8 3.2 2.2 0.9 0.5 -1.3 1.3 1.2 -1.2 0.2 0.8 -2.0
1936 -1.70 -0.37 87 -2.3 -2.6 -2.3 -0.7 -0.4 0.6 -0.2 0.7 2.1 -0.7 -0.4 1.7
1935 -2.09 0.39 87 1.5 1.6 1.6 0.8 -0.5 -0.7 -0.1 -0.6 -0.3 1.8 0.8 -1.2
1934 -1.60 -0.49 85 -0.6 0.4 0.3 -0.7 -0.6 -0.1 -1.7 -0.9 -0.5 -0.9 -0.8 0.2
1933 -1.65 0.05 85 -2.7 -0.8 -0.9 0.6 0.5 1.1 1.4 0.8 0.6 0.2 -0.1 -0.1
1932 -1.96 0.31 83 2.3 0.7 -1.1 -0.1 1.3 0.2 2.4 -0.4 -0.8 -0.1 -0.5 -0.2
1931 -1.28 -0.68 82 0.1 -2.2 -1.0 -1.8 -1.3 -0.8 -3.6 -0.4 0.7 -0.6 1.9 0.8
1930 -1.78 0.51 82 0.6 2.6 1.9 1.5 1.2 0.4 0.3 0.2 0.5 0.0 -0.8 -2.3
1929 -1.57 -0.22 81 -1.2 -0.5 -0.3 -0.6 -1.7 0.3 1.1 1.1 -2.6 -0.2 -0.8 2.8
1928 -1.90 0.33 81 0.7 0.9 0.6 0.0 1.6 -0.1 -1.2 -0.7 2.5 0.4 0.1 -0.8
1927 -2.25 0.35 81 0.0 -2.0 0.0 1.7 -0.4 0.9 1.5 -0.1 -1.7 1.4 1.3 1.6
1926 -1.91 -0.34 79 -0.2 1.7 0.5 -0.6 0.0 -1.4 0.4 0.1 0.3 -1.4 -1.6 -1.9
1925 -1.92 0.02 78 -0.2 -1.3 1.1 -2.2 0.2 0.9 -2.4 -0.3 0.9 0.5 1.8 1.2
1924 -1.60 -0.32 78 1.2 0.4 -3.6 1.4 -1.0 -0.2 2.2 -0.6 -1.0 -0.5 -1.4 -0.8
1923 -1.44 -0.16 76 1.1 -2.9 1.7 -0.9 0.3 -1.1 -1.0 -0.1 -0.4 -0.4 0.4 1.4
1922 -2.00 0.56 74 -3.0 2.6 0.4 -0.3 0.5 2.0 0.4 1.0 1.5 0.8 1.6 -0.8
1921 -1.41 -0.59 72 -0.1 0.4 -1.3 0.8 0.0 -1.7 -1.2 0.1 -0.4 -0.4 -3.0 -0.3
1920 -1.58 0.17 70 1.1 -0.9 -0.8 -0.1 -0.1 0.3 1.4 0.6 0.0 -0.1 0.5 0.2
1919 -1.96 0.38 70 1.4 0.5 1.2 0.5 0.3 0.1 -0.9 -0.4 0.0 0.1 1.2 0.5
1918 -2.14 0.18 68 -0.5 0.1 0.5 0.0 0.7 0.2 -0.2 0.4 -0.7 -0.5 1.5 0.7
1917 -0.91 -1.23 67 -3.3 -1.3 0.6 -0.3 -1.5 -2.6 0.7 -0.9 -0.7 0.2 -3.3 -2.4
1916 -1.29 0.38 67 2.0 0.3 -0.5 0.4 1.2 1.0 0.2 0.3 0.5 -1.1 0.2 0.1
1915 -0.96 -0.33 67 -1.1 0.0 -3.0 0.5 -1.8 0.6 -1.1 -0.2 0.1 1.9 -0.4 0.5
1914 -2.28 1.32 67 2.1 0.6 3.3 -2.0 1.6 1.0 2.3 1.8 2.5 0.0 2.0 0.6
1913 -1.83 -0.44 67 -0.8 -2.2 -1.9 1.0 -0.3 -0.2 -0.5 -1.3 -0.4 0.2 0.9 0.2
1912 -1.43 -0.41 66 0.1 1.4 -0.1 0.0 -0.2 -0.6 -0.5 0.3 -2.2 -0.2 -2.7 -0.2
1911 -2.00 0.58 66 -1.8 1.8 2.0 0.5 -0.1 0.0 0.2 0.4 1.6 -0.4 1.5 1.2
1910 -1.96 -0.04 65 2.0 0.0 -0.1 -1.0 0.2 0.3 -0.4 -0.9 -1.0 1.1 -0.1 -0.6
1909 -2.17 0.22 63 -0.6 -0.4 -0.4 0.5 0.3 0.1 1.5 0.0 1.9 -1.0 1.5 -0.8
1908 -2.00 -0.18 63 -0.8 0.6 0.0 0.1 -0.4 0.9 -0.4 -0.4 -1.0 0.3 -2.2 1.2
1907 -2.17 0.17 63 1.8 -0.5 -0.5 0.0 -0.1 0.0 -0.7 0.9 0.6 0.0 1.8 -1.2
1906 -1.77 -0.40 62 -2.7 0.4 0.2 1.0 -0.1 -1.1 -0.1 1.8 -0.9 -0.6 -1.2 -1.5
1905 -1.74 -0.03 60 3.3 -2.0 -0.2 -2.3 0.5 -0.7 -0.5 -2.5 0.8 0.4 1.5 1.3
1904 -1.67 -0.07 60 -3.1 0.3 -1.9 0.0 0.4 1.7 2.0 0.2 -1.9 0.1 -0.6 1.9
1903 -1.87 0.20 60 2.5 1.1 0.4 2.1 -0.5 0.0 0.4 1.7 0.9 -0.2 -2.3 -3.7
1902 -1.89 0.03 60 -2.2 0.5 1.8 -1.8 0.1 -0.5 -0.3 -1.7 0.2 -0.8 1.8 3.2
1901 -1.94 0.05 60 2.9 -0.2 0.3 0.8 -0.9 0.4 -0.2 -0.7 -0.7 0.6 -1.0 -0.7
1900 -1.68 -0.26 60 -1.5 -1.6 -1.7 -0.4 -0.7 -1.9 -1.2 0.9 1.9 2.0 1.9 -0.8
1899 -1.44 -0.24 59 -1.2 0.0 2.2 1.2 1.2 1.6 -1.0 -1.0 -1.2 -2.0 -2.4 -0.3
1898 -2.06 0.62 59 0.4 1.1 -1.0 0.3 0.0 0.7 1.4 0.2 0.2 1.0 0.4 2.7
1897 -1.76 -0.30 56 0.7 0.0 1.0 -2.1 -0.1 -2.0 0.0 0.2 0.0 -0.6 0.4 -1.1
1896 -1.83 0.07 48 1.2 0.0 -1.3 1.0 -0.3 1.4 1.0 -0.2 -1.0 -0.7 0.4 -0.6
1895 -1.02 -0.82 48 -1.0 -0.3 -1.1 -0.8 0.8 -2.8 -3.5 -0.7 0.2 0.8 -1.3 -0.1
1894 -2.12 1.11 48 0.6 1.9 1.6 1.3 0.7 3.0 1.2 0.7 -0.3 -0.2 1.8 1.0
1893 -1.83 -0.29 44 -0.5 -2.0 0.9 -0.5 -1.0 -0.9 -0.9 0.0 -0.3 0.2 0.0 1.5
1892 -1.56 -0.28 43 0.8 0.0 -3.0 0.4 -0.8 0.8 1.7 0.9 -0.3 -0.4 -0.7 -2.7
1891 -0.72 -0.83 41 -0.9 -2.1 -0.2 -1.7 0.3 -0.9 0.0 -1.0 -0.4 0.2 -0.6 -2.7
1890 -2.17 1.45 36 1.1 2.4 1.5 1.6 1.3 0.4 0.2 0.3 2.9 1.0 1.5 3.2
1889 -1.77 -0.41 31 -1.2 0.9 -0.4 -0.5 -0.9 1.7 -0.9 0.0 -0.4 -0.4 -2.1 -0.7
1888 -1.76 -0.01 27 0.0 -1.6 0.8 0.4 1.2 -0.9 0.6 0.0 -0.3 -1.6 0.8 0.5
1887 -1.65 -0.11 27 0.8 1.6 -0.5 -0.4 -1.0 -0.7 -0.6 -0.5 -1.2 0.1 0.4 0.7
1886 -2.07 0.42 22 0.4 0.0 1.5 0.5 0.6 0.1 1.2 0.5 0.3 0.4 0.2 -0.7
1885 -2.52 0.45 13 -1.2 -0.3 -1.3 0.5 -0.3 0.6 -0.9 1.4 0.1 1.4 2.4 3.0
1884 -1.71 -0.81 13 -0.3 -1.0 0.3 -0.4 0.2 -0.6 -0.9 -1.6 0.2 -2.2 -2.1 -1.3
1883 -1.59 -0.12 13 -1.1 -1.0 -0.4 -0.6 -0.2 0.2 0.4 0.2 0.2 0.5 0.2 0.2
1882 -1.67 0.07 8 2.1 1.2 1.1 1.3 -0.7 -1.1 0.1 -0.8 -1.1 0.6 -1.3 -0.5
1881 -1.44 -0.22 7 0.1 -1.5 -2.5 -0.5 -0.1 1.2 -0.4 0.4 0.5 -0.8 0.7 0.2
1880 -1.16 -0.28 6 -0.6 0.2 0.4 0.1 0.0 -0.9 -0.9 -1.2 0.7 1.4 0.3 -2.9
1879 -1.38 0.23 5 0.3 0.8 0.2 -0.1 0.0 0.4 0.2 0.8 -0.8 -0.5 0.0 1.4
1878 -0.82 -0.56 3 -1.4 -0.8 0.0 -1.0 0.9 -1.4 -0.8 -1.4 0.4 -0.5 0.0 -0.7
1877 -1.12 0.30 2 1.8 0.3 -0.7 0.6 -1.5 3.1 0.4 -1.0 -0.5 0.4 -0.1 0.8
1876 -1.55 0.42 2 0.2 1.0 -0.4 0.9 1.0 -0.3 1.7 1.6 0.2 0.7 -0.4 -1.1
1875 -1.55 0.00 1 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
1855 -1.49 -0.06 1 -2.5 -1.7 0.1 2.4 0.4 0.8 0.8 -1.6 0.6 0.0 0.0 0.0
1854 -1.29 -0.20 1 0.8 2.0 1.3 -0.1 -0.2 -1.1 -2.0 -0.5 -0.9 0.3 -1.3 -0.7
1853 -1.02 -0.27 1 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 -1.1 -2.1 0.0
1852 -1.01 -0.02 1 -1.3 -1.5 -1.8 -1.0 0.0 2.6 1.4 0.2 1.2 0.0 0.0 0.0
1851 -0.37 -0.64 1 1.0 -4.6 -1.4 -0.8 -0.7 -1.9 0.1 1.0 -0.7 -1.2 2.3 -0.8
1850 -0.25 -0.12 1 -0.7 0.0 0.0 0.0 0.0 0.0 1.2 -1.7 -2.0 0.4 1.6 -0.2
1849 -0.23 -0.02 1 0.2 3.8 1.6 -0.8 -0.4 -0.4 -2.5 0.0 1.4 -1.4 -0.1 -1.6
1848 -0.01 -0.22 1 -0.7 0.3 0.3 -0.7 0.0 0.3 0.6 -0.8 -0.3 1.4 -3.5 0.4
1847 -0.63 0.62 1 2.2 0.0 1.3 0.5 2.4 0.1 1.2 0.0 -1.2 -1.4 2.6 -0.2
1846 -0.43 -0.21 1 -3.6 -1.6 -2.5 0.6 -2.4 -1.2 -1.9 1.2 2.8 1.9 0.9 3.3
1845 -0.43 0.00 1 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
For Country Code 210
From input file ./data/v2.mean.inv11.M.dt
-rw-rw-r-- 1 chiefio chiefio 2122537 Mar 1 11:05 ./DTemps/Temps.rM210
-rw-rw-r-- 1 chiefio chiefio 2122537 Mar 1 11:05 ./DTemps/v2.meanM.210
For comparison, here are the relevant years from the simple “average the temperatures” program. Note that in this chart, the “YR” column is on the far right:
Thermometer Records, Average of Monthly Data and Yearly Average by Year Across Month, with a count of thermometer records in that year -------------------------------------------------------------------------- YEAR JAN FEB MAR APR MAY JUN JULY AUG SEPT OCT NOV DEC YR COUNT -------------------------------------------------------------------------- ... 1969 4.1 4.2 6.1 12.4 16.9 19.6 23.8 25.2 22.3 15.8 10.6 5.0 13.8 157 1970 2.6 4.6 4.3 11.6 17.4 19.6 24.3 25.5 22.8 16.9 11.0 5.6 13.8 157 1971 3.4 4.1 6.5 12.2 16.4 20.3 24.7 25.0 21.2 15.6 11.2 6.6 13.9 157 1972 5.7 4.8 7.7 12.4 16.7 20.3 24.2 25.1 21.4 16.7 11.0 6.9 14.4 156 1973 5.2 5.6 6.5 13.8 16.5 19.8 25.0 26.4 21.4 16.2 10.3 4.6 14.3 157 1974 3.3 3.8 6.3 12.7 17.1 20.1 23.2 25.3 21.2 16.5 10.2 5.6 13.8 157 1975 3.5 3.3 6.7 12.8 16.5 20.5 24.4 25.5 23.3 16.9 11.7 5.6 14.2 157 1976 3.3 5.7 7.2 12.1 16.7 20.1 23.2 24.2 20.4 16.0 9.8 5.7 13.7 157 1977 1.6 2.4 7.7 12.7 16.5 20.2 24.9 24.4 22.5 17.5 12.6 7.5 14.2 158 1978 4.0 2.5 6.7 11.9 17.1 21.5 26.2 26.3 22.1 16.3 11.4 7.2 14.4 158 1979 5.0 6.2 7.4 11.8 16.0 21.8 23.6 25.7 22.3 17.7 11.8 7.9 14.8 158 1980 4.0 2.8 7.1 11.5 17.0 21.6 23.1 23.4 21.3 16.5 11.9 5.5 13.8 158 1981 2.1 3.5 7.2 12.3 15.8 20.0 25.1 24.9 20.8 16.0 9.4 6.2 13.6 157 1982 3.6 3.7 8.0 12.0 18.0 19.9 22.8 25.2 21.2 16.8 12.6 7.0 14.2 157 1983 4.4 3.5 7.2 14.5 17.4 19.4 23.3 26.0 22.5 16.3 10.7 5.2 14.2 157 1984 2.2 1.8 4.7 11.3 16.1 21.5 25.2 26.5 21.8 16.0 11.6 6.1 13.7 157 1985 2.1 4.9 7.5 12.9 17.5 19.9 24.7 26.7 22.6 17.0 11.1 5.1 14.3 157 1986 2.3 2.3 6.7 12.7 16.5 20.3 23.3 25.6 22.3 15.3 10.8 7.2 13.8 157 1987 4.3 5.1 7.9 12.3 17.2 20.8 24.8 25.4 21.9 17.9 12.1 7.0 14.7 157 1988 5.5 3.7 7.2 12.4 16.7 20.8 23.1 25.6 22.3 16.4 9.9 6.6 14.2 157 1989 6.5 6.3 8.4 13.4 16.6 19.8 24.0 25.6 22.8 16.7 12.6 7.2 15.0 157 1990 4.0 7.4 9.1 13.0 17.3 21.9 25.2 26.7 23.2 17.7 13.6 8.2 15.6 157 1991 6.0 5.2 8.9 13.9 17.5 22.1 24.5 24.8 23.0 17.8 12.2 8.6 15.4 50 1992 6.6 6.0 9.3 13.7 16.5 19.9 24.2 25.1 22.1 17.4 12.4 8.7 15.2 50 1993 6.6 6.8 8.0 12.5 16.9 20.3 22.7 23.9 21.7 16.8 13.6 8.1 14.8 58 1994 5.4 6.2 7.5 14.2 18.2 20.9 26.2 27.1 23.5 18.7 13.5 8.7 15.8 58 1995 5.5 5.8 8.6 13.5 17.5 19.8 24.8 26.4 22.3 18.7 11.9 7.1 15.2 50 1996 5.5 5.0 8.3 11.4 16.6 21.0 24.5 25.2 22.2 17.6 12.9 8.0 14.8 58 1997 5.7 6.1 9.4 13.7 17.7 20.9 24.6 25.2 22.1 17.1 13.8 8.9 15.4 58 1998 5.4 7.0 9.6 15.5 19.0 20.8 24.6 25.9 23.6 19.4 12.9 8.9 16.0 54 1999 6.1 5.8 9.3 13.4 17.7 21.5 24.2 26.4 24.3 18.4 13.2 7.9 15.7 54 2000 6.5 4.6 8.2 12.9 17.7 20.9 25.3 26.2 23.1 18.2 13.0 7.8 15.4 53 2001 4.6 5.4 8.5 13.7 17.8 21.3 25.4 25.3 22.1 18.0 12.4 6.9 15.1 54
So we see a long series of about 13.x to 14.x that shifts to about 15.x at the transition.
An interesting question to explore would be why the January data step from 2.x to 6.x has some wobble around 4.x/5.x in the 1988-1990 area. For 1990, there will have been many stations that had monthly values drop out mid year as the thermometers were dropped. ( I still make monthly averages from the surviving months and an annual average from all the data). But why does 1989 jump up? Is it an artifact of missing months handling? An adjustment change? Random?
This pattern argues for something such as an adjustment history change in 1987-1989 that is simply a coincidence of timing with The Great Dying of Thermometers in 1990 or some kind of displacement of the data by one year at the transition as I accumulate the totals (an “off by one” issue). But it will take a fair amount of work to sort it out in detail. Is the slightly divergent timing due to coincident process changes at NCDC? Due to an accidental artifact of how the two programs compute things? Due to monthly drops prior to the entire thermometer drop? Who knows. I’ll be working through the details of the code and GHCN to see what I can figure out. Access to the Japanese monthly average data directly would be very helpful. We’ll see where it ends up. But “CO2 it ain’t”.
Yes, this is a giant “Dig Here!”. Were I a researcher at the Japan Meteorology Agency, I would be inspecting my data as collected and comparing it to what NCDC has created from it. Then I would be sharpening both my pen and my sword… and deciding which to use and on whom. Saipuku or… But that’s just me and my romantic notions about Japan.
Further Back In Time
If you take a moment and look past that 1990 watershed, you find a very stable series. One of the wonderful things about Islands it that they are fairly stable places. The geography is generally fixed. It does not have too much variation and is usually limited in longitude and latitude such that not much can be changed. Altitude is about all there is to fool with. Furthermore, you have a large body of water around them. While it is possible to have an Arctic Express run over an Island, the effects are moderated. Almost by definition you are getting a natural smoothing and filtering of the temperatures of the planet. And we see that in the temperature history of Japan.
Fairly stable, but with some minor impacts from change and from year to year weather shifts. But in the end, the Pacific Ocean wins out, and stability is returned. So we see a lot of -1 C +/- about 1/2 C until we drop down to one lone thermometer that is a little warmer than now. It would be interesting to know if the 1840’s were recorded as a warm time in Japan. To know if this is an artifact of that particular thermometer being in a warm place, of if the Japanese record is reporting a core truth down to the last thermometer standing.
Musings from the Chiefio 
“The Smith Effect”: ah yes, the smith works at a forge, doesn’t he?
In case that comes over the wrong way: it’s not the chiefio whom I suspect of forging numbers. Quite the contrary.
REPLY:[ I rather liked the ambiguity, but I’m strange like that ;-) And remember that the Smith often just makes what the customer orders… -E.M.Smith }
‘But I’ll leave it for others to decide if The Smith Effect is named for Fort Smith or “some other Smith” ;-)
Or both.
I am now starting to get some understanding of ‘Climate Logic’ (where the present can change the past and suchlike).
Using such logic – I am amazed at the prescience of the founders of Fort Smith who were able to name a place after you, before you were even born.
I can’t pretend to completely understand all of what you have posted here, but on the face, it seems you have made a very strong claim (again!!!) about the validity/quality of the data being used in the climate science debate.
Keep up the great work.
OT:
One of my favorite history classes in college was the study of Japan and the effect that Admiral Perry had on the Samurai (and the results on Japanese culture).
OT II:
I LOVE SUSHI/SASHIMI !!!
The temperature jump around the time of the great dying of the thermometers may be real: SST temperatures in the west Pacific jumped around 1988-1990. Have a look at this Bob Tisdale post from last year:
http://bobtisdale.blogspot.com/2009/05/revisiting-misunderstandings-about-pdo.html
This post by Bob might also interest you: http://bobtisdale.blogspot.com/2009/06/part-1-of-comparison-of-gistemp-and-uah.html (eyeballing it, it seems to me that the gap between GISS and UAH for Asia started to open up wider from around 1990)
The ‘Smith Effect’ et al being slow roasted here: (snip)
Perhaps you can clear up some conceptions, miss or otherwise.
REPLY: [ I’ve looked at it. Not all that interesting. Various trolls let run rampant in the comments too. I can’t spend my time fixing everyones broken ideas, all I can do is ‘my stuff’. Some folks will get it, some won’t. Such is life. -E.M.Smith ]
Hello Chiefio,
Magic Java has requested that I invite you to guest post your response to the analyses by Zeke, Tamino, ClearClimateCode and Roy Spencer, each of which has investigated the effect of the 90’s loss of thermometers on any bias in computed temperature anomalies. I think this is a good idea, and would like to invite you to guest post, if you are interested.
If you do accept, I would actually like to invite you to do two guest posts. In one, I would ask you to answer some very specific questions of my own devising. These may be questions you do not wish to answer– as they will put you on the spot. The other would simply invite you to post your response to the analyses of others. My most specific interest is to see and understand why you think the loss of thermometers has resulted in an actual warm bias in products like CRU, GISSTemp and NOAA’s surface temperature products for large areas of land. That is: I am interested in the effect averaged over areas as large as the US , the full Northern Hemisphere or the Southern Hemisphere.
With this in mind, I extend and invitation and hope you will take it up either at my blog, or at yours.
Thanks,
Lucia Liljegren
(Sorry if this double posts. One of my readers said my previous try didn’t show.)
REPLY[ You hit a SPAM filter key word. I’ve deleted the duplicate posting that was O.T. on the blood pressure thread. I’ve responded in email. The ‘hyper short form’ for all the other curious folks is “I’m really busy and not all that interested in food fights over Other Peoples Hypothetical Cows. I think raw temperature bias can leak through GIStemp since it uses temperatures for calculations prior to the anomaly step and then does anomalies “Basket A in Time A vs different Basket B in time B” and once you have apples to oranges comparisons, you are open to bias leakage. Further, since Spencer has now extended his time length and found divergence between Jones and raw, I see no issue between his findings and mine. -E.M.Smith ]
Hey Chief,
I know it’s O.T. but I wanted to pass along an interesting (to me at least) article on the off chance that you might have a few spare moments and be looking for an interesting read. Note: Ars is a techy site (if you don’t know it already) and unlike El Reg quite Pro-AGW. To bad they can’t see the correlation between Medical and Climate research…
We’re so good at medical studies that most of them are wrong
http://arstechnica.com/science/news/2010/03/were-so-good-at-medical-studies-that-most-of-them-are-wrong.ars?utm_source=rss&utm_medium=rss&utm_campaign=rss
Pingback: The Blackboard » Timeline of “The march of the thermometers” meme