Climate Audit

In a previous post, I’ve observed some oddities in connection with the dating of Wahl and Ammann 2007 and with Schneider’s obfuscation when asked to explain how an article supposedly accepted on March 1, 2006 could cite an article that had not even been submitted until August 2006. (BTW, I note that Journal of Climate uses the term “In final form” in its timelines, which, applied to the Wahl and Ammann case, would preclude any dates prior to August 2006, the date of submission of Ammann and Wahl 200x, which is cited in Wahl and Ammann 2007, and, more likely, preclude any dates prior to the June 2007 acceptance of Ammann and Wahl 200x – a date which appears to have triggered the actual acceptance of Wahl and Ammann 2007.)

Why would they go to this trouble? Why wouldn’t they just put the acceptance date for Wahl and Ammann 2007 as June 2007, concurrent with the acceptance date of Ammann and Wahl 2007, which unblocked the publication process?

To answer this, we need to look at the IPCC Publication Deadlines, where we shall see a compelling and incontrovertible link between Wahl and Ammann timelines and attempts to at least appear to meet IPCC publication deadlines.

IPCC Working Group 1 Publication Deadlines

Three main milestones were set out in the IPCC Publication Deadline timetable:

May 10-13, 2005 – Literature to be cited will need to be published or available in draft form by this time (the date of the Second Lead Author meeting).
Dec 13-16, 2005 – Literature to be cited will need to be published or in press by this time (the date of the Second Lead Author meeting).
Late Feb 2006 – the TSU must hold final preprint copies of any unpublished papers that are cited in order that these can be made available to reviewers. This means that by late-February 2006 [the date for preparing the Second Draft] if LAs can not assure us that a paper is in press and provide a preprint we will ask them to remove any reference to it.

There are several different versions of the WG1 Publication Deadline Timetable. An early version is here; The most recent version is online here. Also see the IPCC document entitled “Deadlines for literature cited in the Working Group I Fourth Assessment Report”, preserved as of Apr 30, 2006 (and unchanged as at June 21, 2006).

The above deadlines were well publicized and guided the behavior of climate scientists. For a huge study like AR4, for better or worse, there have to be some cut-off dates or the study would never end. For example, Judy Curry has mentioned here her understanding that there was an end-2005 deadline, an understanding that presumably guided her publication schedule.

Wahl and Ammann 2007 Chronology

The first deadline cited above was May 10-13, 2005. On May 10, 2005, Ammann and Wahl submitted Wahl and Ammann 200x to Climatic Change and Ammann and Wahl 200x to GRL -see here – and issued a press release on May 11, 2005 here . The submission date on the published version of Wahl and Ammann 2007 is May 11, 2005.

The second deadline was December 13-16, 2005 and “literature to be cited will need to be published or in press by this time”. The
UCAR website states that Wahl and Ammann had been “Provisionally Accepted” on Dec. 12, 2005 – on the eve of the expiry of the second IPCC deadline. Has anyone ever seen a paper report the date of its “Provisional Acceptance”? Clearly “provisional acceptance”, whatever it is, is not the same thing as being “published or in print”. Wahl and Ammann had clearly missed an important deadline – a deadline that Judith Curry and others were adhering to.

The end of February 2006 was a drop-dead date by which “the TSU must hold final preprint copies of any unpublished papers that are cited in order that these can be made available to reviewers. This means that by late-February 2006 if LAs can not assure us that a paper is in press and provide a preprint we will ask them to remove any reference to it.” The UCAR website states that Wahl and Ammann was supposedly “accepted for publication” on Feb 28, 2006, this third date, like the other two, being on the eve of an IPCC deadline. So there is an incontrovertible pattern.

The timing of the Climatic Change decision was very curious, to say the least. GRL turn times are very short. The GRL decision on the contingent GRL submission would be available in 2 weeks; within the four corners of journal administration, there was no reason for Climatic Change to make a hurried decision ahead of GRL – especially given that Schneider had already been burnt once by Ammann on his GRL submission. Why wouldn’t Schneider just wait a couple of weeks? Or is it possible that UCAR/Ammann, with an eye on IPCC deadlines, put pressure on Schneider to “accept” the article before the end of February? Sure seems likely. And I wonder if Schneider’s acceptance still had some strings attached – something that seems highly likely given the subsequent delays. Ammann and Wahl 2007 was published over a year after any other Chapter 6 journal citation.

Ammann placed Wahl and Ammann 2007 online in early March 2006. It was brought to my attention and I immediately made a number of posts – see here on March 10, 2006 and there are a number of contemporary posts, including ones where I placed my 2005 Review and correspondence online. This new version included a table showing that the MBH verification r2 for the AD1400 step was 0.018 (we had reported 0.02). Despite this confirmation of at least one of our results (and they confirmed many of our results), Wahl and Ammann did not withdraw their allegation that all our claims were “unfounded”.

On March 16, 2006, GRL notified Ammann and myself that the Ammann Comment had been rejected, with Famiglietti stating:

The simple message, with which I concur, is that there is nothing significant that is new here, so that there is no need to publish the present exchange.

Despite this GRL rejection over two years ago on March 16, 2006, the UCAR webpage on this matter states (as of today):

We are working on an update of this site and a cleaned-up version of the code since the Wahl and Ammann paper is accepted by Climatic Change and in press. Once GRL has reached a decision, we will also post the appropriate codes and illustrations here. (Note, the current version is fully functional, and no results will change.)

They were obviously very quick to announce the Climatic Change acceptance, but not so quick to announce the GRL rejection. Securities commissions would make mincemeat of mining promoters that did this.

A version of the webpage was scraped on Feb 5, 2006. The only change between that version and the present version is the insertion of the above paragraph. The last recorded change to the webpage (as of today) was on March 25, 2006. Ammann’s webpage presently states:

Caspar Ammann
Last modified: Mon Mar 25 13:57:32 MDT 2006

Thus, there was one and only one change on this webpage between Feb 5, 2006 and today: the insertion of the above paragraph. A change to the webpage is attested on March 25, 2006. The only plausible conclusion is that the above paragraph was inserted on March 25, 2006. If so, the paragraph was untrue and overtly deceptive at the time it was published on the UCAR website. I would welcome any interpretations of this information (and will post any such interpretations in this paragraph).

Returning to the IPCC schedules. On April 7, 2006, according to the IPCC Schedule, the Second Order Draft was to be made available to reviewers and the 8-week review period for expert and Government reviewers was to start. I downloaded the Wahl and Ammann version then online at the WG1 website for unpublished papers; my download is dated Apr 7, 2006 on my computer, which I take to be the date that I downloaded the pdf. (On Apr 9, a CA reader noted that online that the stale version was still online at WG1). The pdf document properties stated that the pdf had been created by Oyvind Paasche, a Chapter 6 Contributing Author, on March 10, 2006. Curiously this version was identical to the May 10, 2005 version that I had received as a reviewer (converted to pdf form.) So the version online at WG1 (and presumably the version available to Chapter Author Paasche) was not the “accepted” version or even the September revision, but the original submission which failed to disclose the failed verification results. Recall the requirement “the TSU must hold final preprint copies of any unpublished papers that are cited in order that these can be made available to reviewers.” The Wahl and Ammann version then held by the WG1 TSU was not a “final preprint” – even the March 2006 version was not a “final preprint”. But the situation was even worse, the TSU was not even in possession of a current version of the paper.

TSU presumably became aware of the problem as a result of CA attention. Instead of placing a notice of the substitution on their website and sending a notice of the substitution to Reviewers so that any Reviewers that had downloaded the stale version could download the current version, they quietly substituted the new Wahl and Ammann version at the WG1 website without any notice whatever. I don’t know the exact date on which they made the substitution. I downloaded the newer version on June 2, 2006, which, again by coincidence, was the end of the 8-week review period for expert and Government comments.

The IPCC schedule provided that Review Comments be collated and sent to Chapter Lead Authors on or about June 16, 2006.
This schedule was closely adhered to. At the present website, the front page of the online Review Comments states that these are from “Batch AB (15 June 2006)”.

A couple of IPCC reviewers (with whom you are familiar) noticed the non-compliance of the Wahl and Ammann article (and a couple of other articles) with published IPCC publication deadlines and objected to the inclusion of these articles. See, for example, comments 6-1115, 6-1116, 6-748, 6-1158, 6-750, 6-1215 and 6-735.

Review Comment 6-1158 (B 29:41 29:41) stated:

Wahl and Ammann 2006 did not meet several publication deadlines. Is it fair to use this study when other studies also not meeting publication deadlines were not used? It was not accepted by December 13-15. TSU did not have a preprint by late February. The version available for review was not the same as the accepted version – in particular, the version made available omitted critical information that MBH98 failed cross-validation r2 and CE statistics. [Stephen McIntyre (Reviewer’s comment ID #: 309-119)]

or 6-1115 which stated:

It seems very unwise to me to waive IPCC WG1 policies on publication guidelines, especially for lead authors. For example, Osborn and Briffa 2006 did not meet the December deadline for being published or in print; it was not even mentioned in the First Draft nor was it available from TSU as part of the First Draft process. Other citations in the chapter did not meet the December deadline for being published or in press as at the December draft meeting (Osborn and Briffa 2006; Wahl et al 2006; Wahl and Ammann 2006; Hegerl et al “accepted”); several did not meet the February drop-dead date for providing TSU with a preprint (Wahl and Ammann 2006; Hegerl et al “accepted”). The version of Wahl and Ammann 2006 as accepted differed dramatically from the version provided to TSU for both the First Order and Second Order Drafts, notably in respect to the inclusion of their calculation of MBH verification statistics confirming the results of McIntyre and McKitrick showing failure of MBH verification statistics that had previously been denied. [Stephen McIntyre (Reviewer’s comment ID #: 309-12)]

The CLA Replies were, in a way, predictable. But there’s something very interesting in the wording of these rejections that I didn’t notice when the Review Comments became available in 2007. See if you notice.

Rejected, guidlines used for preparing the draft have been followed and new guidlines do not pose problems 6-1115)

Papers cited are within the guidlines for in press papers 6-1116)

Rejected- the citation is allowed current rules. 6-748

Rejected- the citation is allowed under current rules. 6-1158

See response to comment 6-1158. (6-750).

Rejected – revised deadlines mean that these papers are citable. 6-1215

Rejected -… all papers cited are within current IPCC publication deadline rules. 6-735

Notice that they didn’t contest the validity of the Review Comments under the Publication Deadlines that existed at the time that the Review Comments were made. Watch the pea under the thimble as the Chapter Authors purport to justify these Replies.

A more typical sort of Review Comment was the suggestion by authors that their own 2006 publications be cited. Here’s a typical example by D. Stone (9-476):

9-476 I conceitedly suggest adding the following recently accepted papers: Stone, Allen, Selten, Kliphuis, Stott. 2006. Stone, Allen, Stott. 2006. A multi-model update on the detection and attribution of global surface warming. Journal of Climate; Crooks, Allen, Stott. 2006. An update on the detection and attribution of observed temperature change – a space-time separable approach. Journal of Climate

Another example Review Comment 9-816:

9-816 A 49:1 49:2 This has been more thoroughly tested in: Pall, Allen, Stone. 2006. Testing the Clausius-Clapeyron constraint on changes in extreme precipitation under CO2 warming. Climate Dynamics.

Two weeks after the delivery of the above Review Comments to Chapter Lead Authors, pursuant to prior IPCC schedules, the Fourth Lead Author meeting took place in Bergen, Norway from June 25-30. The schedule states:

This meeting considers comments on the second order draft and revisions to produce the final draft start immediately afterwards.
June 25 chapter meetings
June 26-28 LA4 meeting
June 29-30 Meeting of the TS/SPM writing team

Most institutions make minutes of their meetings and I presume that IPCC Lead Author meetings are no different. It would be interesting to see the minutes of this meeting.

On July 3, 2006, IPCC WG1 sent an email to reviewers (which I received), a scraped version of which is onlinehere, a document the name of which indicates that it was written on July 1, 2006.

Guidelines for inclusion of recent scientific literature in the Working Group I Fourth Assessment Report.

We are very grateful to the many reviewers of the second draft of the Working Group I contribution to the IPCC Fourth Assessment Report for suggestions received on issues of balance and citation of additional scientific literature. To ensure clarity and transparency in determining how such material might be included in the final Working Group I report, the following guidelines will be used by Lead Authors in considering such suggestions.

In preparing the final draft of the IPCC Working Group I report, Lead Authors may include scientific papers published in 2006 where, in their judgment, doing so would advance the goal of achieving a balance of scientific views in addressing reviewer comments.

However, new issues beyond those covered in the second order draft will not be introduced at this stage in the preparation of the report. Reviewers are invited to submit copies of additional papers that are either in-press or published in 2006, along with the chapter and section number1 to which this material could pertain, via email to ipcc-wg1@al.noaa.gov, not later than July 24, 2006.

In the case of in-press papers, a copy of the final acceptance letter from the journal is requested for our records. All submissions must be received by the TSU not later than July 24, 2006 and incomplete submissions can not be accepted

A short turn time of July 24, 2006 was required because the IPCC schedule required that Chapter Lead Authors return annotated responses to all comments on the second draft to TSU by August 4 (which would be the approximate date that one can ascribe to the Chapter Lead Author Replies presently online). Given the well-publicized deadlines – ones that Judy Curry believed to have still existed – by what authority were these new guidelines issued? Are there minutes of the meeting? Did someone make a motion? Did someone second the motion? Was anyone opposed?

I presume that the Chapter Author Replies presently online date from approximately August 4, 2006. In response to Stone’s first suggestion, they replied:

The papers that are in press have been included.

Consulting the papers, the first timeline and second timeline for the two suggestions read:

Manuscript received 10 June 2005, in final form 21 April 2006)

Manuscript received 13 June 2005, in final form 24 April 2006)

So these references fall within the July 2006 variance – they were suggested by reviewers, they did not introduce a new topic and they were in final form. One wonders whether the TSU observed the protocol set out in the July letter to reviewers and obtained the required letter of acceptance from the Reviewer. IPCC policy requires that all written reviewer comments be placed in an open archive. The July 2006 correspondence would obviously be “written comments” under this definition, but wasn’t placed online with the other comments.

It does seem possible that Chapter 9 authors did check such matters. The other Stone suggestion referred to above online here was turned down as follows:

This paper is not citable here, since its status is ‘submitted’.

Its publication timeline shows that it was accepted on July 21, 2006 – a date which did not entitle it to be cited.

25 May 2005 / Accepted: 21 July 2006 / Published online: 30 August 2006

So you can see why Ammann and Wahl are so determined to preserve the fiction of a March 2006 acceptance date even Wahl and Ammann 200x was not “in final form” and “editorial components” were “incomplete”. I’ve browsed through the IPCC references and so far I’ve been unable to locate any other article that has a comparable timeline to Wahl and Ammann 2007. I’ll post on this separately.

The July variance is very limited and did not permit Chapter Lead Authors to include any 2006 article whatsoever. The policy required that a Reviewer “suggest” an article and “submit” it to the TSU. In-press articles had to be accompanied by the acceptance letter. In most walks of life, this sort of variance would be very strictly administered to avoid favoritism.

However, as we’ve seen above, the Chapter 6 Authors applied this rather specific variance to qualify Wahl and Ammann 200x,
Osborn and Briffa 2006 and Hegerl et al 200x.

In many contractual situations, they wouldn’t have been allowed to do this. The following argument would be made: had the Chapter Authors excluded these articles in the Second Draft, as IPCC policies required, Reviewers would be entitled to suggest their citation to clarify an issue under the July 2006 policy. But the July 2006 policy could not be used to salvage prior breaches of the policy by Chapter Authors, especially where the policy breach was in favor of their own article (Osborn and Briffa 2006) or other IPCC Chapter Authors (Hegerl et al 200x; Wahl and Ammann 200x).

Note that if Wahl and Ammann was supposedly qualified under the July policies, then a Reviewer was obligated to submit a copy of the Wahl and Ammann 200x acceptance letter to the IPCC TSU. This would be a “written comment” and should be on file. Will they have one? I doubt it, but we might as well find out.