Climate Audit

It looks like there is a huge cock-up in the von Storch and Zorita Comment, which completely screws up their simulations. We alluded to a defect in their methodology in our Reply, but I didn’t realize just how big a defect it was.

Von Storch and Zorita said the following:

MM05 performed a Monte Carlo study with a series of independent red-noise series; they centered their 1000 year-series relative to the mean of the last 100 years, and calculated the PCs based on the correlation matrix. It turned out that very often the leading PCs show a hockey stick pattern, even if the data field was by construction free of such structures. This finding was recently confirmed by others (F. Zwiers, personal communication).

We observed the following about this in our Reply.

VZ said that the MM05a Monte Carlo study calculated PCs on the correlation matrix; in fact, following MBH98, we used the decentered data matrix in our simulations. From this, we surmise that VZ calculations were not done by SVD on the decentered data matrix; this will have a material impact on their yield of hockey sticks.

I just realized how "material" the impact of this defect in their implementation of the hockey stick algorithm: it eliminates the effect totally!! Think about it for a moment – if you change the centering of the data set, you don’t change the correlation matrix. So if you do principal components on the correlation matrix of the short-centered data, you get exactly the same eigenvectors (weighting factors) as correctly centered data. You only get different eigenvectors if you do SVD on the short-centered data directly and take the left matrix, as Mann did. When you re-read the above VZ quotation narrowly, they have not shown that they have replicated the algorithm, only that Zweiers had confirmed the effect. Because we were happy that von Storch and Zorita seemed to confirm the Artificial Hockey Stick effect, we assumed that they had somewhat implemented the hockey stick algorithm and didn’t fully worry through this issue in our Reply, beyond the above sentence.

It would have been nice to have fully realized this at the time of our Reply, because von Storch has been nice to us and I would have given him a heads-up on the issue if I’d realized it. On the other hand, we’d asked to see their code to see what they were doing and didn’t get anywhere. We even had to argue with GRL to get the above sentence in. It is possible that the VZ description of their algorithm is inconsistent with their actual algorithm, in which case, the problem with their pseudoproxies is the more germane issue. But, if the VZ source code is consistent with the description in their article, the entire von Storch and Zorita simulation is totally screwed up. This won’t amuse realclimate.

Update (Friday): The following question needs to be explained as well: if the correlation matrices are the same, then isn’t this proving too much – i.e. wouldn’t VZ then get identical results either way? Presumably, in their implementation, they standardized on the short segment, not just short-centering, but also short-scaling. The short-segment standard deviations for each series will differ from the full-series standard deviation. So the correlation matrices are the same, but the weights from the eigenvectors are applied to versions which have been divided by slightly differing standard deviations. It is the difference between these standard deviations that accounts for the very slight difference in the two VZ runs, rather than the impact of the MBH98 method. This slight difference in standard deviations would be a minor glitch, if that were the problem. Indeed, short-segment standardizing in this form would, on the face of it, slightly discriminate against hockey stick shaped series, although the effect is slight. (This is doubtless why MBH98 used something called detrended standard deviations, which do not downweight hockey stick shaped series. The Hockey Team is serious about hockey sticks and every little edge is used.)

Update (Saturday): von Storch has sent me an email saying that they used covariance matrices in their PC calculations and defended his calculations. I’m considering the email and will post a more definitive response after this consideration.