I’ve spent an inordinate amount of time working up a practical emulation of Mannian CPS and have uploaded a function to do this. While Mann has made a decent effort to archive code and intermediates, as others have noted, the code itself is chaotic and hard to figure out. Added to this are the “stupid pet tricks” – the allocation of proxies to more than one gridcell if they are on a fence-post, the unexpected (and unmentioned in the running text) 1 degree displacement of all gridcells to the south and the repetitive smoothing of truncated proxies at each step. Plus the archived reconstructions are spliced versions, so it becomes that much harder to reconcile.
However, with the help of UC’s Matlab runs and key assistance from CA readers (especially Roman’s filtfiltx function), I now have a working function manniancps which is presently located at http://www.climateaudit.info/scripts/mann.2008/utilities.txt. It requires the downloading of a considerable amount of data which can be carried out by executing http://www.climateaudit.info/scripts/mann.2008/collation.cps.txt (requires the package signal for Mannian smoothing).
A “verbose” version of the function will also yield various intermediates. These are compared to UC Matlab versions for the AD1000 step NH iHAD in the script http://www.climateaudit.info/scripts/mann.2008/replication.cps_AD1000.txt.
For comparison of reconstructions, I’ve used the SH iHAD reconstruction since, although the archived versions are spliced, the peculiar Mannian splicing procedure preserves the AD1000 network in this network, rendering it more useful for benchmarking than other networks.
Here is a comparison of my emulation of the SH iHAD cps series to the archived version.
Top – SH iHAD CPS – top: emulation vs archive (splice in this case); bottom – differences.
The algorithm sets three parameters to control for “stupid pet tricks” – smoothmethod; outerlist and lat.adjustment. If these are set to “sensible” procedures i.e. smoothing the entire proxy record once and using this smoothed record throughout; allocating proxies to only one gridcell and not displacing the gridcells, one gets a somewhat different NH reconstruction, as shown below.
Figure 2. Comparison of NH iHAD CPS with and without “stupid pet tricks”
The “pet tricks” have a not inconsiderable effect on the relationship between the 1930s and 1980s.
To analyze substantive issues like the impact of using Tiljander proxies upside-down, one has to wade through this stuff. It’s taken a while to get the emulation to stand still, but I should be able to now do these sensitivities in fairly short order. (The EIV recons remain a bit of a black box, unfortunately, and will take some time to parse.)
The differences arising from “stupid pet tricks” are, in some cases, a not inconsiderable proportion of the claimed uncertainty. At present, I have no idea how Mannian “uncertainties” were calculated. I asked Mann et al 2008 reviewer for information on this topic, and, in his best traffic cop manner, he told me to “move on” – which I take to mean that he doesn’t have a clue how they were calculated.
Climate Audit 
15 Comments
I’ll get some time tomorrow to try the code out, it took a minute to look at it all together today. What a huge amount of work!
The little jump in the difference at 1100 looks like a ringing filter to me. I’ve seen that on an oscilloscope a thousand times. Not that it is what’s happening here.
Jeff Id, #1, says,
Or maybe it is. Mann likes to use the Butterworth filter. This is a filter whose primary virtue is that it can be implemented in electronics with non-ideal components. There is no reason to use it in statistics, where mathematics allows one to build ideal circuits. But if, like Mann, one uses the Butterworth filter’s mathematical representation anyway, one is likely to discover quirky electronic side effects.
Re: Hu McCulloch (#2),
Actually, maximal flatness in the passband is a pretty strong reason to use it in the digital domain as well. The problem with Mann’s use of this filter would be a problem for ANY IIR filter, since they have an infinite impulse response. In other words, they ring. Since he’s using a 10th order filter, it will ring even more than a lower order filter.
Mark
The ringing at 1100 comes from the effect of one of Mann’s stupid pet tricks. The archive is a splice of different steps, while my emulation is just the AD1000 step. The AD1100 network has exactly the same components as the AD1000 network but Mann re-butterworths the data starting in 1100 – so what you’re seeing here is the impact of the stupid pet trick through the difference between the splice and the AD1000 network. I’m 99.999% sure that it’s not due to a problem in how I’ve emulated it. At some point I’ll do the splice and I think that the match will be even closer.
Excellent Work. I don’t understand most of it, but still, it’s excellent! 😛
Re 3
So what you say is essentially that the details of Mann’s climate reconstruction ca 1100 AD is not due to anything in the proxy data, but to a clumsy handling of a mathematically inappropriate filter function?
#5. The filtering causes odd effects. Different filtering methods lead to somewhat different results in past periods. Why the data is filtered at all is still unclear. It would be going too far to say at this point that the filtering is a cause celebre, as opposed to a stupid pet trick.
Re: Steve McIntyre (#7),
Mann read a paper somewhere and decided that since signal processing people do it, he might as well do it, too. Unfortunately, signal processing applications that employ filtering typically also have known signal and noise relationships, and filters are specifically designed so that the signal is passed and only noise is attenuated/excised.
Mark
Re Mark T #8 and Steve #7: Actually, a type of “Kalman smoother” (a filter which looks in both directions) wouldn’t be unreasonable for a reconstruction, provided it were based on empirical variance ratios.
Our discussions here of calibration tend to think in terms of calibrating one date at a time, in isolation from all other dates. However, the instrumental data is, to a first approximation, roughly a random walk, so that adjacent dates are more alike than distant dates, and adjacent calibration data and even instrumental data can tell us a lot about any given date in question.
An ideal calibration would then take into account the empirical variance of the instrumental random walk, plus the emprical variance of the calibration equation, and then do a type of Kalman smoother under the assumption that the unobserved reconstruction temperature is continuing this random walk. The near end can even be pinned down with “perfect” certainty (relatively speaking) at the oldest instrumental date.
I’m still mulling how to do this with Thompson’s CC03 ice core data, which is only a Z-mometer as it stands. The problem is somewhat complicated by the fact that even given the variances, the slope uncertainty makes the pointwise reconstruction the ratio of two normals, and then the variance uncertainty also has to be integrated out, presumably numerically. But it’s not insurmountable, I think.
Re: Hu McCulloch (#9),
Yes, random walk or any other prior for the signal. CCE + Kalman, that’s what we need. In published dendro work ICE gets trough (Briffa98, see http://www.climateaudit.org/?p=4475#comment-314389 ,
implicit assumption is iid signal, not random walk), and even variance matching + butterworth smoothing (Mann). And Briffa’s variance adjustment shows that they are not even interested in taking this topic seriously 😉
BTW, I think Butterworth is at least approximate solution to some Wiener filter problems.
Kalman filters aren’t really “filters” in the same sense as we’re discussing with the IIR being used. They are adaptive, and thus derive their coefficients directly from the data being filtered.
Mark
#9. Hu, it may be interesting to look at speleothem O!8 concurrently with ice core O18.
Oh, and I don’t disagree that such a smoothing method wouldn’t be unreasonable.
Mark
I’m having a bit of trouble with the Require Rcompression line in the collation file.
require(“Rcompression”)
Is there an R package which has this already built in?
1. I don’t think using terms like “stupid pet tricks” is helpful to getting neutral observors or even the objects of criticism to look at your in process work. In addition, it plays to the hoi polloi who don’t follow the technical arguments and just want to agree with you, like some sort of Jerry Springer fans. That and it’s way too cute. Just use a neutral phrase to describe the methodology setting options.
2. you should have (at least) 4 settings. The grid square displacement and the only one gridcell are separate issues.
3. The 1930s impact is interesting. Good catch.
4. I wonder how much RE is impacted by the methodology choices (minor, I assume, but just curious).
5. Is your example here only the 1000 AD proxies or does it include the more recent stuff? Just want to make sure that you can emulate the more complicated Mann treatments. Why not emulate figures from his paper?
6. If you have questions on how the uncertainty was calculated, ask Mann. I would have told you to move on too, were I the reviewer. Also, the whole issue deserves it’s own post, or at least it’s own paragraph! Don’t throw it in at the end.
3 Trackbacks
[…] “Emulating Mannian CPS“, Steve McIntyre, Climate Audit, 2 December 2008 – The struggle continues to get […]
[…] “Emulating Mannian CPS“, Steve McIntyre, Climate Audit, 2 December 2008 – The struggle continues to get “hockey stick” Mann’s computer to code to work. Only then can Mann’s work be replicated. Odd that it appears in peer-reviewed journals; one wonders what “reviewed” means when the code does not run. […]
[…] “Emulating Mannian CPS“, Steve McIntyre, Climate Audit, 2 December 2008 – The struggle continues to get “hockey stick” Mann’s computer to code to work. Only then can Mann’s work be replicated. Odd that it appears in peer-reviewed journals; one wonders what “reviewed” means when the code does not run. […]