This isn’t as ‘cheeky’ a question as it might seem.
All the statistical manipulation I’ve seen done on temperatures tends to presume they have a Standard Normal Distribution and that it is a valid statistical operation to compute a mean. While this might seem reasonable for a single thermometer in a single place ( but even there can ‘have issues’ ) as soon as you start doing an arithmetic mean over a geographic field of thermometers spread over 1200 km (as is done in codes like GIStemp) you are making the implicit assumption that the mean is defined.
For those of us who had some, but not extensive, statistics, that is a natural assumption as we spent months (years?) doing various kinds of problems all universally based on a Standard Normal Distribution. But there are other kinds of distribution…
Why this matters is that it is a property of the Standard Normal Distribution that allows the use of the Central Limit Theorem in the production of those numbers with astounding precision based on lousy input data.
The normal distribution is considered the most prominent probability distribution in statistics. There are several reasons for this: First, the normal distribution arises from the central limit theorem, which states that under mild conditions, the mean of a large number of random variables independently drawn from the same distribution is distributed approximately normally, irrespective of the form of the original distribution. This gives it exceptionally wide application in, for example, sampling. Secondly, the normal distribution is very tractable analytically, that is, a large number of results involving this distribution can be derived in explicit form.
Notice some of the ‘requirements’ of those statements. “Mild conditions”. “Large number”. “Random variables”. “Independent”. “From the same distribution”. “Approximately normal”. But per Hansen et. al. various temperatures in a region (up to 1200 km) are NOT “random variables” but are in fact co-variant. We have about 1280 thermometers currently in use globally in the GHCN last I looked. That means that any 1200 km ‘field’ is far less populated. That is NOT a ‘large number’ statistically. Thermometers near a large ocean have very different behaviours from those inland a couple of hundred miles in mountains or deserts. Those in the snow have different behaviours from those in the dry valleys. They are not “the same distribution”. Are conditions “Mild” in Antarctica or the Sahara? I don’t know if they meet the requirements of statistically “mild” or not. Right off the bat we have some concerns showing up in the use of the Reference Station Method and the choice to use the mean of a set of temperatures for statistical manipulation of other temperatures and the infilling of missing data into various specific locations and the larger “grid / boxes”.
But wait, there’s more…
A bit further down in that wiki we find the first hints that all might not be well:
Quantities that grow exponentially, such as prices, incomes or populations, are often skewed to the right, and hence may be better described by other distributions, such as the log-normal distribution or the Pareto distribution. In addition, the probability of seeing a normally distributed value that is far (i.e. more than a few standard deviations) from the mean drops off extremely rapidly. As a result, statistical inference using a normal distribution is not robust to the presence of outliers (data that are unexpectedly far from the mean, due to exceptional circumstances, observational error, etc.). When outliers are expected, data may be better described using a heavy-tailed distribution such as the Student’s t-distribution.
Do temperature readings have outliers? Well, yes! A.Watts found that there were a suspicious number of high temperature values reported from arctic and similar cold locations and traced it back to the encoding used (where "M" is used for "minus" meaning a negative value. 30 C is quite an outlier from -30 C. As one example.)
So we have an immediate flag that perhaps the Standard Normal Distribution is "less than right"…
The next little problem I note in passing is that temperature data is not "randomly drawn" from a distribution. Temperature data is selectively drawn from population centers (that are themselves non-randomly distributed, being biased to valley floors, flat plains, and places were water and land meet such as bays, harbors, and major rivers. More recently, as cities form where 2 or more modes of transportation intersect, we have more cities around airports, sea ports, and where railroads intersect those.) At present, the majority of our land temperature data is drawn from airports. By definition, airports are selectively located where weather conditions are most conducive to regular and safe operations.
All those kinds of "issues" have already been explored (and the temperature data found skewed and wanting due to them – Urban Heat Island effect and Airport Heat Island effect), though not in the context of their statistical validity. Here we start to note that perhaps the location selection bias itself might invalidate the statistical assumptions that underlay the use of an extended averaging process to increase the precision of temperatures. (Presently reducing a 1 F minimum error band in the raw data to 1/100 C of precision in the mean. IMHO False Precision; for just these kinds of reason.)
While I’ve explored in prior postings that point that such an averaging process can remove random error in measurements, it can NOT remove systemantic error. (Such as the systematic error introduced with the conversion to the MMTS thermometers. Both changing location to be closer to buildings due to the need for power and communications cables; and the “adjustment” to “remove a cooling bias” that was really locking in a slow warming trend from aging of paint on the prior Stevenson Screens). I’ve not asked the more basic question of “is that use of the mean and the central limit theorem statistically valid?”
This shows that there are already many reasons to doubt that the mean of temperature measurements is used in a valid way. But might there be more?
What if “mean” is undefined?
While looking into some things involving IR and “backradiation”, I was looking up pressure broadening and the nature of the distribution of energy between different species (such as water, ozone, nitrogen oxides, carbon dioxide, sulphur oxides) in the air. During that, I ran into the point that the nature of the broadening curve changes are NOT a standard normal distribution. Instead, they are a different kind of distribution. One for which the mean is not defined.
Its importance in physics is the result of its being the solution to the differential equation describing forced resonance. In mathematics, it is closely related to the Poisson kernel, which is the fundamental solution for the Laplace equation in the upper half-plane. In spectroscopy, it is the description of the shape of spectral lines which are subject to homogeneous broadening in which all atoms interact in the same way with the frequency range contained in the line shape. Many mechanisms cause homogeneous broadening, most notably collision broadening, and Chantler–Alda radiation.
So pressure broadening, which we know is driving the nature of the interaction between species and the distribution of the infrared (and all other photons too) is not a standard normal distribution. It is a Cauchy Distribution. As all of the AGW thesis rests on the notion of IR redistributing heat via just this physics of molecules, I think “this matters” to the basic assumptions of AGW.
The Cauchy distribution, named after Augustin Cauchy, is a continuous probability distribution. It is also known, especially among physicists, as the Lorentz distribution (after Hendrik Lorentz), Cauchy–Lorentz distribution, Lorentz(ian) function, or Breit–Wigner distribution. The simplest Cauchy distribution is called the standard Cauchy distribution. It has the distribution of a random variable that is the ratio of two independent standard normal random variables.
So here we see the kinds of things that cause this sort of distribution, and why they are so common in physics. The question to ask is simply: Are temperatures a function of two (or more) independent random variables?
I would assert “yes”. (Though the question of ‘are they standard normal’ I leave open…)
I was in Dallas once when it moved some 50 F in one day. A cold front from Canada swept over us. It is pretty well accepted that air motion is chaotic. While, in the short run, we can make modest weather predictions, the randomized chaotic nature means that the ability to forecast rapidly breaks down over a few days to weeks. So, as temperature data in GHCN is a ‘monthly mean’, there is one factor that already has a strongly random component in the ‘less than a month’ scale. Then there is cloud cover. Any given patch on the ground on any given day may have, or not have, a cloud overhead. IMHO that’s a second largely random term. The existence of just these two, IMHO, raises grave doubts that the temperature data are a standard normal distribution.
Now, layered on top of that, there are several non-random processes that change the data as well. There are seasonal cycles. There is the day / night cycle. There are ocean cycles and tidal cycles. Many of those push the weather to display what looks like “stochastic resonance”. And what was one of the properties typically leading to a Cauchy distribution? “Forced resonance”. IMHO there is a whole can of worms to open just in this one space. We know that there are resonance effects all over the weather / climate / ocean systems; and that those show up as temperature changes. To what extent do these matter? We don’t know. While I would assert they are THE dominant force changing temperatures, that’s just a naked assertion. Yet the ENSO / AMO / PDO and many other “Modes” and “Oscillations” of the air and water are main drivers of temperature changes over the scale of years to decades (and perhaps to centuries for longer cycles).
What does THAT mean? To me, it makes it highly likely that the temperature change distribution follows a Cauchy type distribution and highly unlikely that they follow a Standard Normal type distribution. That, then, brings up one small point about some of the other kinds of distributions. (A point that I vaguely remembered being mentioned in the first few weeks of my statistics classes just before we promptly “moved on” to the Standard Normal Distribution and never really looked back other than a brief dip into the Student-T distribution… but only remembered AFTER being reminded by this text…)
The Cauchy distribution is often used in statistics as the canonical example of a “pathological” distribution. Both its mean and its variance are undefined. (But see the section Explanation of undefined moments below.) The Cauchy distribution does not have finite moments of order greater than or equal to one; only fractional absolute moments exist The Cauchy distribution has no moment generating function.
Because the integrand is bounded and is not Lebesgue integrable, it is not even Henstock–Kurzweil integrable. Various results in probability theory about expected values, such as the strong law of large numbers, will not work in such cases.
Oh Dear! Hearing “pathological” and “undefined” are not things you want in the statistics that underlay the entire edifice of a Global Average Temperature statistic.
For me, this raises (confirms?) some strong doubts about the validity of the statistical manipulation done to the temperature data. All over the place it gets adjusted, homogenized, in-filled (interpolated / extrapolated / whatever) and anomalized and has precision extended out to 1/100 C in ways that look to me like they depend on assumptions about the kind of distribution in the data (in particular, Standard Normal ) that are unwarranted.
But that’s as far as I can take this muse. My statistics background ended with Student-T in an undergraduate course. This would take a much more skilled statistician that knows these other distributions and what can be done with them “cold”. That’s not me.
So if you happen to know a good skeptical statistician (or be one!), I think this would be a good “Dig Here!”. To simply ask “Are the necessary prerequisites for using means and averages present in the actual distribution?” and “Do we even know what the distribution might be?” To me, on the limited evaluation (example above) that I’ve done, temperature data looks like it is not a standard normal distribution type; and it looks like the “climate science” processes I’ve seen just assume that it is one. An assumption that is highly likely to be in error, IMHO.