Mann on Source Code

Mann told Antonio Regalado of the Wall Street Journal that he would not be “intimidated” into releasing source code for MBH98.

Here is an account of correspondence with Mann and with the U.S. National Science Foundation:

Even before publication of MM03, we politely requested clarification on issues in MBH98. This was a source of controversy in late 2003. Here is a record of correspondence with Mann which we made available some time ago.

After publication of MM03, Mann argued that MM03 contained an incorrect implementation of a stepwise principal components procedure (which was not documented in MBH98) . Details of this procedure have continued to drift in, with the first listing of the number of PC series retained in each calculation step/tree ring network combination provided in the July 2004 Corrigendum SI. This listing was inconsistent with prior information.

In August 2004, through Nature, we became aware privately of claims that a variation of Preisendorfer’s Rule N had been used to determine the number of retained PC series. This claim was published in November 2004. (We have not been able to verify actual application of this criterion, as actual numbers are impossible to replicate. See Was Preisendorfer’s Rule N Used?

In any event, immediately after we learned of the previously undocumented stepwise procedure, we asked to inspect MBH98 source code so that we can completely reconcile results and avoid this type of dispute. Attached is our correspondence after MM03, which obviously cannot be construed as a form of “intimidation”, but as an entirely proper request.

Subsequently, we located some Fortran code at Mann’s FTP site for the calculation of tree ring principal components. Although this code is only a very small fraction of the total code, it contains a procedure which was materially misrepresented in MBH98 and which additionally is not statistically valid. We reported on this in MM05 (GRL) and MM05(EE).

As I’ve pointed out in various postings on Replication, it is impossible on the present record to replicate important steps in other parts of MBH98.

After our unsuccessful attempts at obtaining source code, we asked the U.S. National Science Foundation for assistance. This was also unsuccessful. The correspondence is here.

We also made attempts with Nature and I’ll get to describing this on another day.

3 Comments

  1. Spence_UK
    Posted Feb 27, 2005 at 11:32 AM | Permalink

    The link "our correspondence after MM03" seems to be broken – I tried changing the comma to a full stop but it still didn’t seem to work?

    Steve: Fixed

  2. Jeff Norman
    Posted Feb 27, 2005 at 10:09 PM | Permalink

    Are the following mistakes in Margaret Leinen’s letter in the original correspondence or were these transcription errors?
    – "… general premise of tour(sic) letter …"
    – "… scientific data obtained with public funks(sic)."
    – "… a wealth of paleoclimatic data that its(sic) organized by scientific subspecialty …"
    – "In an instance for which you may not yet be aware,"
    – "… that addresses concerned(sic) raised by you …"

    Jeff
    Steve: My bad. except for the second-last. I’ve corrected the typos. It came in a non-extracting pdf and I transcribed it quickly for a collation. I’ve put up the pdf for comparison, linked from the transcribed version.

  3. Rob Kaper
    Posted Mar 8, 2005 at 10:25 AM | Permalink

    It saddens me so many people take black box research for granted. Obviously peer review doesn’t guarantee perfection nor does secrecy rule it out, but any claim that isn’t verifiable is simply not scientific.

2 Trackbacks

  1. […] Mann Correspondence — MBH March 7, 2010 shewonk Leave a comment Go to comments Here’s a CA post that refers to correspondence between McI and Mann re data. Mann told Antonio Regalado of the Wall Street Journal that he would not be “intimidated” into […]

  2. By More on Requests for Data « Climate Audit on Nov 11, 2010 at 4:02 PM

    […] not be "intimidated" into releasing his algorithm, I posted the following overview on our requests for source code to show that there had been no "intimidation". Such language from these guys – Mann […]