NOAA is first of the three main indices to be off the mark with June 2009 at 0.617 deg C, bouncing off the relatively low values of 2008. Given that the data is essentially common to HadISST, this is unsurprising.
The difference between RSS and NOAA/HadCRU values is interesting in terms of the Big Red Spot (enhanced tropical troposphere temperatures: RSS should be going up faster than NOAA/HadCRU, not the opposite.) Here is raw RSS and NOAA data: note that the reference periods are different! (I left the original reference periods to separate the lines a bit better.) You can see how NOAA surface is gaining on RSS T2LT rather than the opposite.
Figure 1. TRP temperatures to June 2009. NOAA (1961-1990) vs RSS (1979-1998).
I’ve done quite a bit of experimenting recently with an interesting program called strucchange (Achim Zeilis). I originally experimented with this program in connection with hurricane data: to test the supposed regimes of Holland and Webster. I looked at it again in connection with the new USHCN algorithm (in case the presently secret USHCN adjustment program was ever disclosed – it uses breakpoints methods as well.)
More recently, I applied it to various crosscuts of the TRP satellite and surface data: RSS vs UAH, Land vs Ocean and various cross-profiles. Many interesting results that I’m assimilating.
Today, I’ll show one such crosscut: Tropical RSS vs NOAA, crosscutting Land, Ocean and All.
As you can see, this particular algorithm finds “significant” breakpoints in these crosscuts. Aside from what the algorithm finds, visually there’s a big difference between the Land and Ocean patterns that seems like it would be hard to justify in climate terms. (The same thing happens with UAH vs NOAA, it just looks different.) There are also significant breakpoints between UAH and RSS.
In most cases, the breakpoint location can be plausibly associated with either the start of one satellite or the end of another. The tricky thing about this association is that there are lot of stitches in the satellite record and the mind is prone to finding associations. More on this below. For now, take a look at the graphic.
Figure 2. TRP NOAA minus RSS. Difference series centered on zero.
But in this case, when one examines the literature on satellite adjustments, I think that there’s pretty good reason to anticipate that breakpoints could occur at satellite switches. Complicating matters is that the literature also reports issues with “drift”.
Also the most cursory examination of the satellite literature shows that it is highly statistical in concept. In some case, there is limited ground truthing between satellites, so they end up having to estimate the adjustment – a statistical operation.
My take on this is that there are going to be at least 6 adjustments that need to be estimated. In some cases, there is both a step adjustment and a drift adjustment. If one admits the possibility of statistical error into the procedure, then you no longer have an AR1 error model in trend estimations (something that I’ll show in another post.) It looks to me like there are 5 or 6 or more step adjustments, which generate highly significant AR1 coefficients, but the underlying process is different and more complicated. This would be a big and interesting project.
In the bottom panel of the above graphic, the increase of NOAA relative to RSS T2LT over land in the past 10 years is particularly consistent. Again, Big Red Spot Theory predicts the opposite. At this point, I’m not inclined to view any of this as “falsifying” Big Red Spot theory, but, more likely, as evidence of “drift” or “bias” in both surface and satellite records. There is certainly food here for people who think that the surface land record is affected by measurement bias.
However, I’m far from convinced that the satellite records are revealed truth. It seems quite possible to me that quite different satellite trends could emerge if there were a couple of inter-satellite adjustment errors. I don’t know right now how one would estimate the potential magnitude of the adjustment errors, but, as soon as one introduces potential step adjustment errors, it becomes pretty hard to estimate trends. More on this on another occasion.
Climate Audit 
31 Comments
I’m going to be away for a few days (I’m doing some gold exploration business in northern Ontario). So please behave.
No piling on, venting, editorializing.
Good work on this, Steve. I hope you work with Roy Spencer and John Christy. I am certain they will be much more cooperative in supplying data and methods than usual.
I was thinking about the problem of finding a good source for ground truth for the satellite measurements. It seems to me that the launch ranges around the globe may have collected some useful data. On the day of a launch, and often on other days as well, atmospheric data must be collected to see if the atmosphere is within the “nominal” range for launch. I know we use these balloons for wind data, etc., but don’t know if they specifically include temperature measurements. Of course, these measurements would have their own uncertainties as well. At any rate, given the launch rate of Kourou, Baikonur, and Cape Canaveral and that they are a drastically different latitudes, perhaps there is enough data to do a sanity check of the overflight measurement vs the balloon measurement. Just a thought.
Steve two questions:
1) You wrote “You can see how NOAA surface is gaining on RSS T2LT rather than the opposite.” Is that simply an eyeball test (which I can’t see) or is there some computation involved?
2) How does the “strucchange” routine differ from the old Chow test and the various updates of it (e.g. Andrews, Donald W. K., “Tests for Parameter Instability and Structural Change With Unknown Change Point,” Econometrica, 61(4), 821–856, 1993.)?
I recall Jeff Id posting an article by Jeff C. on the topic of cloud cover measurement and satellite transition. It is posted here at tAV:
Jeff Id’s posting also indicated it was posted here at CA originally. Jeff C.’s charts look similar to the ones here.
Perhaps one of the Hollywood types would lend out a spare G5 for contemporaneous calibration of sensor data against (gasp) an honest-to-goodness observation.
The US Dept of Agriculture in the 1990s sponsored some substantial surveys for getting ground truth against satellite data to calibrate their efforts to encourage precision agriculture. These surveys collected data on soil moisture and ground cover, along with the relevant GPS points. I assume temperatures were also collected.
It would probably take some work to pull this together in an acceptable form, but the data has been gathered.
It is my understanding that if we assume that wind measurements in the troposphere are more accurate than temperature measurements, then there is a lot of warming somewhere up there we have missed. (But see Pielke Sr., R.A., T.N. Chase, J.R. Christy, B. Herman, and J.J. Hnilo, 2009: Assessment of temperature trends in the troposphere deduced from thermal winds.) Though apparently nowhere near enough to establish even a small red spot ….
I note from the graphs above that when the anomaly is up or the trend is warm, the NOAA figures don’t mind agreeing with RSS but when it’s cooling per RSS, the NOAA figures are reluctant to fully join the dip. Maybe surface temps are less variable (i.e., as per the supposedly meaningless UHI effect) or maybe some NOAA correction algorithm resists valleys but not peaks (or maybe if you plant enough weather stations on concrete and rooftops…).
Also, I am unsure of the current ontological status of the “red spot” qua “fingerprint”. I was under the impression that RealClimate.org had spoken ex cathedra and relieved the faithful of any duty to regard excess tropospheric warming as a canonical element of AGW but I can’t find the link.
Re: George Tobin (#8),
More opportunistic post hoc re-framing of the hypothesis? When you find the link, please post it.
WRT the accuracy of satellite trends:
http://www.agu.org/pubs/crossref/2008/2007JD008864.shtml
Seems to me that the evidence is that RSS is OVER adjusted-and naturally the discrepancy between UAH and the surface data will be even worse.
Have you tried variability magnitude adjustment on RSS before taking the lapse rate? For change points that will matter because, at large events like the 97-8 El Nino, discontinuities can be introduced. I recommend taking John Christy’s 1.3 factor and dividing the satellite values or multiplying the surface. In fact, if you look closely, the reason for the “breakpoint” around 99 is pretty obvious.
You are being brave venturing into the Northern Ontario bush in mid July.
We once had a Black Fly land at our airport and we pumped 500 gallons of JP4 into that sucker before we realized it wasn’t a 747. Then it fly off, dragging one of the re-fueling guys away into the bush.
Still haunted by his screams, but being so out numbered – I figure about 999,999,999,999,999,999,999,999,999 to 1, there was nothing we could do to rescue him.
So make sure you have lots of 12 gauge Muskol.
Bender:
I did in fact misspeak.
Instead of “I was under the impression that RealClimate.org had spoken ex cathedra and relieved the faithful of any duty to regard excess tropospheric warming as a canonical element of AGW”
I should have said
“I was under the impression that RealClimate.org had spoken ex cathedra and relieved the faithful of any duty to expect to see any evidence of excess tropospheric though it remains a canonical element of AGW
Here is the RC link
1) On the one hand, Real Climate is more adamant than the IPCC on the appearance of the Hotspot:
Whereas the IPCC says there is such a thing as an AGW troposphereic fingerprint:
The incomparable lucia discusses the matter in more detail here:
2) But, of course, the fundamental problem is that hot spot has not appeared. Worse, skeptics published papers pointing out the discrepancy between the models and the data (see, e.g.,: International Journal of Climatology of the Royal Meteorological Society [DOI: 10.1002/joc.1651]. David H. Douglass, John R. Christy Benjamin D. Pearson & S. Fred Singer)
3) So, RC said the following at link provided above:
It’s OK that there is no observable greater warming in the troposphere because
(1) We don’t really expect to see it over a mere 30 year period.
(2) El Niño events
(3) volcanic eruptions
(4) failure to add uncertainty ranges to observations (presumably sufficiently large to create an exonerating overlap with model projections)
(5) failure to adjust radiosondes and
(6) failure to given the models an even bigger uncertainty range (presumably sufficiently large to create an exonerating overlap with observations).
(7) probably lots of other stuff too and skeptics aren’t very smart. (I made that last one up.)
In other words, it is a scientific certainty that (1) the hot spot is real but (2) we don’t really expect to see any evidence of it therefore it is not really a crisis of faith that it does not appear to be there. Which is why I characterized the matter in the manner I did. I should have been more precise.
Another paper which also finds a discontinuity in RSS (in fact, around the time that they go from NOAA-11 to NOAA-12):
http://www.agu.org/pubs/crossref/2007/2005JD006881.shtml
The issue of splicing together satellites is interesting and I’m glad to see it get brought up, since there is little discussion of this issue in most places.
Whatever the root cause, the magnitude of discrepacies shown here is surprisingly similar to what is seen in comparing independent small-sample averages of US temperatures. I doubt that anybody is entitled to claim knowledge of regional temperatures more closely than ~0.25K.
Interesting that those breakpoints approximately (or maybe exactly, in some cases) correspond to satellite switch-over times.
Re: Ryan O (#15),
The early satellite sensors at least in this series were designed to be good for 2 years. Maybe that’s why there’s scant information about overlaps.
Touche’, Mr. Tobin.
I have also noticed something odd with both GISS, UAH and HadCRUT, but maybe nothing. Using the climate explorer I generated a time series of temperature deviation over the south island of New Zealand for each series and compared them, GISS yieldest the greatest warming trend, followed by HadCRUT then UAH, not unusual, However, I also pooled individual GISS station data for NZ south island and the trend calculated is very small compared to the large warming trend obtained from the gridded data on climate explorer (0.03c/decade vs approx. 0.3c/decade). I might be working it out wrong, I am not expert on statistics, but if what I have done is ok, than something does seems odd. The raw station data shows either very mild warming, no change or slight cooling. I am not sure how GISS turn their own station data into a significent warming trend? or why the UAH data also shows a decent warming trend?
Re: stumpy (#18), It’s plausible that the data needs such adjustments and even possible that GISS’s particular effort has done a good job. But we’ve observed here before that quite a few instances have GISS making adjustments which are decidedly…odd? This adds to the anecdotes.
Re: stumpy (#18),
I agree with Andrew. We have seen some strange anecdotes but I think this one trumps them all. Quite often, the adjustments are a good percentage of the overall trend (sometimes even equaling the trend). I do not think I have seen an anecdote in which the adjustment increased the trend by the power of 10. I hope you follow up on this.
Re: stumpy (#18),
Geert Jan and colleagues at KNMI have done a remarkable job in a short time and are to be congratulated. However, it is inevitable that some errors have slipped through the cracks and Geert Jan simply does not have the time or resources to do the quality control he would like to do. Often, the errors occur before the data reach him. So step 1 in the use on KNMI data is to do a thorough comparison with any other data that are available and relevant, as you seem to have done.
NASA GISS out for June 2009 and showing a +0.63°C anomaly.
Klotzbach, P.J., R.A. Pielke Sr., R.A. Pielke Jr., J.R. Christy, and R.T. McNider, 2009: An alternative explanation for differential temperature trends at the surface and in the lower troposphere. J. Geophys. Res., submitted.
Click to access R-345.pdf
Re: Jos (#23), I was GOING to get to that, thanks. 🙂
Re: Andrew (#24), Don’t know what the current status is though. It has been on the site as “submitted” for quite some time now. I’ll ask Roger Sr.
Re: Andrew (#24), Still under review.
Satellites may not be perfect, but at least the raw data is available and they tell you how they splice it together.
Plus there is onboard calibration and cross checking. More recent satellites require less corrections for orbit and daylight crossing, and less corrections is always good.
It is normal there are differences when you change how the splices are done, and what adjustments are applied. But that gives you an idea of the robustness of the result. It is part of the openess and debate.
It not anything like as messy as the arbitory adjustments in the ground level record. The USA may have been audited, and found wanting, but the rest of the world largely unaudited, and less likely to be pretty in the light of day.
Re: Sean Egan (#28),
You say that the satellite record is “not anything like as messy” as the surface record. Have you read the primary literature on the construction of satellite temperature records? If you have, I’m not sure that you’d say this.
Sean Egan (#28),
The conversion of the satellite radiances to temperatures requires the inversion of an integral equation, i.e., it is an ill posed process. The problem is usually “solved” by an iteration process that only partially works in the presence of ground based measurements. Thus if there are errors in the ground measurements, there will be errors in the associated satellite retrievals. And hen there are clouds present, the retrieval process is highle questionable.
Jerry
Re: Gerald Browning (#29),
This is definitely the case where the object is to construct a complete temperature profile from the measured radiances using the eleven channels (4 to 14) of the AMSU that cover the pressure range from 1000 to 1 mbar. The literature I’ve seen says that you need a “first guess” of the temperature profile, not just the surface temperature. The implication being that you really need a balloon sounding somewhere in the vicinity at some time close to when you want to use the satellite data, with vicinity and close not being well defined in what I read.
Please correct me if I’m wrong on this, but I’m not at all sure that UAH and RSS go through the full process of extracting a complete temperature profile at every grid point on the planet for their measures. UAH is known to use combinations of readings at different angles of the same channel to correct for the overlap in altitude of the different channels while RSS, if I understand correctly, uses linear combinations of the different channels at just one angle. There is also lots of averaging in both space and time. RSS still doesn’t use readings from AQUA, so they need to do orbital drift correction as well.
DeWitt Payne (#30),
Consider the noninvasive tomographic process used in medicine, e.g., a spiral CAT scan. There an integral is also being inverted , but the scan consists of a multitude of radial paths thru the body. In fact the number of scans is so high that the radiation dose is of concern (this is from personal experience). Thus if there are sufficient paths at different angles, the large scale structures can be recreated, but not the small scale structures that are more sensitive to retrieval errors.
Typically in satellite retrievals the inversion is done thru only one vertical column and is thus of questionable value, especially when clouds are present.
When I had Sylvie Gravel (manuscript available on this site) use satellite temperature data without any radiosonde or surface data to help with the inversion of the integrals, the forecast results using the satellite temperature data were a disaster. On the other hand the wind data from radiosondes and aircraft
was sufficient to produce the same accuracy as the complete set of observational
data, i.e., the satellite data was superfluous (as expected from mathematical theory).
Jerry
One Trackback
[…] June 2009 and the Big Red Spot- Värmen är tillbaka! Eller? Läs även kommentarerna till Steve McIntyres inlägg. […]