Just got back from Washington a few minutes ago (I think that the presentations went very well) and saw this press release from Wahl and Amman (link). The points appear to be ones that have been posted up at realclimate before, which we’ve fully considered and, in my opinion, don’t lay a glove on our criticisms. Here are the links as sent to me. I’ll post up some more comments after I’ve had a chance to look at it.
Boulder,CO May 10, 2005 Two new research papers submitted for review by Eugene R. Wahl (Alfred University) and Caspar M. Ammann (National Center for Atmospheric Research) reproduce the recently criticized Mann-Bradley-Hughes (MBH) climate field reconstruction, and invite researchers as well as the public to use the code for their own evaluation of the method. Wahl and Ammann highlight the robustness of the MBH method against numerous modifications. Their detailed analyses (first presented publicly at the American Geophysical Union fall meeting, San Francisco, 2004 and at the American Association of Geographers annual meeting, Denver, 2005) reveal that the highly publicized criticisms of S. McIntyre and R. McKitrick against the “Hockey Stick” climate field reconstruction of MBH (Nature, 1998) are unfounded. Conclusions of temperatures during the 15th century rivaling the late 20th century climate are found to be without statistical and climatological merit, as is the alleged identification of a fundamental flaw that would significantly bias the MBH climate reconstruction towards a hockey stick shape. Wahl and Ammann address each criticism raised by McIntyre and McKitrick (Energy and Environment 2003, 2005 and Geophysical Research Letters 2005) and find that: “⠠High 15th century temperatures are only achieved with statistical models that don’t pass validation tests, and thus lack both statistical and climatological meaning. Using mean 20th century climatology would provide a more successful reconstruction than the results put forth by McIntyre and McKitrick. “⠠If Principle Component analysis of North-American tree ring data is applied appropriately (i.e., with retention of all the relevant climate information contained in the tree ring data), climate reconstructions very similar to MBH are achieved, independent of the reference period applied. This is confirmed by an analysis leading to the same result if all individual tree ring series themselves are included. Code for Reconstruction based on Mann-Bradley-Hughes: http://www.cgd.ucar.edu/ccr/ammann/millennium/MBH_reevaluation.html “Real Climate” elaboration on various issues: http://www.realclimate.org Contact information: “⠠Caspar Ammann, National Center for Atmospheric Research: 303-497-1705, ammann@ucar.edu “⠠Eugene Wahl, Alfred University: 607-871-2604 wahle@lafred.edu
Climate Audit 
6 Comments
I was originally optimistic that the source code might reveal clues as to the method applied by Mann (quirks and all) but having briefly scanned the code it looks to me like all the important decision steps are missing and the code only performs the last stage – for example, the PC’s to use etc. appear to be hard-coded into the control data file.
It would be interesting to see the paper, I bet they don’t mention R-squared anywhere! So their spurious RE statistics remain as meaningless as ever. Plus they point out that the high 15th C. carries no significance, which makes it appear as if that was once claimed, which of course it never was…
Seems the hockey team won’t let the thing die a natural death, they are determined to prolong the agony. I notice on realclimate they are maintaining the line that other studies make this one irrelevant (so why defend bad science so vehemently?) and that there are still very few graphics showing the Mann curve without the instrumental record overlaid…
Spence, I’ve been working through the code, which changes this quite a bit from usual Hockey Team situations. They don’t report the R2 anywhere and it isn’t even calculated in the code as far as I can tell. I’ll calculate the R2 for the AD1400 step tomorrow. The argument appears to be: they claim a higher RE with the bristlecones in than with the bristlecones out without any other justification.
Their whole posture on “robustness” seems very evasive to me. Mann et al. warranted that their reconstruction was robsut to the presence/absence of dendroclimatic indicators (even though they knew from the CENSORED bristlecone pine sensitivity study that it wasn’t). The warranted robustness was one of the reasons for the wide acceptance of MBH. Now everything in W-A merely confirms that their reconstruction is sensitive to the presence/absence of bristlecones.
I’ll bet dollars to doughnuts that the W-A 15th century reconstruction fails the R2 test or else they would have reported it. Spence: do you agree with this: if a process supposedly has a true correlation with temperature of (0.5, 0.4, 0.3 almost anything you name), then the probability of yielding an R2 of ~0 in a 48 year verification test is going to be vanishingly small. Indicatively the lowest true correlation that I can get that is consistent with a R2 of 0 is about 0.05. At this R2, there is no meaningful confidence interval that can be estimated.
If a process fails an R2 test, then the hypothesis that it has a correlation to temperature is falsified. AS to why it has a spurious RE statistic, that is only of forensic interest. Steve
So am I right in assuming that this paper one agian relies heavily on the bristlecone pine data, which shows no correlation to local temperatures in the instrument record.
Steve,
I did try to take a “red team” position on the R-squared question, trying to come up with a reason why it could be ignored, but it is difficult to do so. R-squared can be viewed as a form of “variance explained” and can be seen as a very similar metric to the eigenvalues generated by the PCA – but, critically, the eigenvalues from the PCA respond to an offset mean (such as that caused by the decentred PCA method used by Mann), and are therefore not a pure measure of variance explained in this case. To this end, the R-squared value should be a better representation of variance explained than the PCA eigenvalues. I don’t see how you could ever justify a claim that the eigenvalues are meaningful but the R-squared value isn’t.
Looking closely at the words, in many ways their paper simply confirms the dependency on the Mann method on the bristlecone pines. Looking through their scenarios (a)-(d), with the BCP you get a “significant” response without them you do not – although they try to word the paper such that it appears this is caused by the standardisation procedure. They do not separate out this aspect (which would have been trivial to do) so what they have published on the web is inconclusive on this point (we will have to wait and see what the paper says I guess). They then try to twist the argument with a straw man, claiming that the low verification stats achieved by your paper is somehow a reflection on your work, rather than a weakness of the MBH methodology. Quite sly, although we know the hockey team do have a certain panache when it comes to spinning out a good story. It is just a shame they seem to prefer “fiction”.
Spence.
My first reaction was that it seems like many of the things I’ve seen on Ammann’s website are re-hashes (for example, getting Mann’s results without using PCs at all).
I also didn’t appreciate the hard-to-discern spaghetti graphs.
The one thing that struck me the most was the noteworthiness applied to the fact that M&M’s 15th century results didn’t pass statistical validity tests. It seems to me that this was already known http://www.uoguelph.ca/~rmckitri/research/trc.html “…In our E&E article we showed that the MBH98 reconstruction has high early 15th century values, as shown in the Figure below, after applying two changes: (1) using the archived version of the Gaspé tree ring series rather than the version with ad hoc editing by Mann et al.; (2) using exactly the same number of series as MBH98, but with standard centered PC calculations rather than the data mining method of MBH98. However, neither reconstruction has any statistical significance…” The inclusion of this item reads like propoganda to tell people that M&M are wrong and MBH98 is right about the 15th century, when in fact MBH98 had the same statistical problem.
Yeah…it’s hilarious and shows the low level of ascientific tendentiousness that they stoop to, that they criticize the statistics of “your reconstruction”, when what you are showing is that their reconstruction is NOT ROBUST.
The more and more of this that goes on, the more I think that the failed physicist is not that much of a scientist, but more of a promoter.
One Trackback
[…] to Steve McIntyre’s Climate Audit blog two papers were written and sent out for publication. These papers were somewhat controversial […]