Core Count in Phil Trans B

The Yamal reconstruction was introduced in Briffa 2000, a survey paper that did not include elementary information like core counts. As a result, users of the Briffa 2000 Yamal reconstruction (including Mann and Jones 2003, Moberg 2005, Hegerl 2007; D’Arrigo 2006, IPCC 2007, etc…) used it without any knowledge that the core counts did not meet RCS standards. In his recent online article, Briffa said that the closing portion of the Yamal results should be used “cautiously”. A Delayed Oscillator reader observed at Delayed Oscillatorthat this caveat should have been made clear in Briffa’s previous papers:

Briffa should have made clear in his papers that the post 1990 reconstruction was based on very few trees, and so should be “treated with caution”, as he explained in his recent web post.

Instead of agreeing with this obvious point, Delayed Oscillator argued that Briffa et al 2008 “shows sample size for each chronology”. Briffa et al 2008 would constitute notice only to users of this data after 2008 (such as Kaufman et al 2009), but does not constitute notice for users up to and including IPCC AR4. I wish that climate scientists would simply concede this sort of unwinnable point and focus on points that are interesting ones. If they don’t understand that notice in 2008 cannot be effective notice in 2000, it’s hard to have a sensible discussion.

But even the disclosure of Yamal sample size in Briffa et al 2008 is far from satisfactory. Here is their Table 1 reporting the number of samples during periods specified in the Table. In the -200 to 2000 period, Yamal is listed as having 611 samples, nearly double the number of samples for Avam-Taimyr (330), even though Avam-Taimyr had over 100 samples in 1990, while Yamal had only 10.

In the Phil Trans B measurement archive provided in Sept 2009, there are indeed 330 cores in the Avam-Taimyr sample, but for Yamal there are only 41% of the reported “611” samples. Phil Trans B Figure 3 shows core counts by year and looks like it uses the correct core count for Yamal (252 not 611.) The scale for core counts is inconsistent between panels, so that the relatively low closing counts for Yamal are not as clear as they might be.

If it weren’t for the incorrectly reported number of 611 samples for Yamal, Delayed Oscillator might be able to argue that Figure 3 constituted notice to post-2008 users and that Kaufman et al 2009 had satisfactory notice. However, under the circumstances, it is surely more than a little embarrassing that Briffa et al 2008 incorrectly reported the Yamal sample size as 611 rather than 252. I wonder where the figure of 611 came from??

52 Comments

  1. Gary
    Posted Nov 5, 2009 at 9:46 AM | Permalink

    Steve.

    I note the fuzzy view of the pasted data in the above post. A good trick to improve on this I have learned is to convert the subject document to PDF, scale up the view in the PDF document, copy the bit you need and paste into the new file. You can then scale down the pasted data and it usually will result in less fuzzy results. If this is of use, enjoy. If it is an annoying bit of detail, snip away.

    • benpal
      Posted Nov 5, 2009 at 10:40 AM | Permalink

      Re: Gary (#1), click on the image for a larger view.

    • Jon
      Posted Nov 5, 2009 at 11:43 AM | Permalink

      Re: Gary (#1),
      or use png instead of gif so that the artifacts don’t show up as much.

      • Earle Williams
        Posted Nov 5, 2009 at 12:50 PM | Permalink

        Re: Jon (#4),

        Minutia suitable for Unthreaded, but it is the JPEG format that creates artifacts. For charts and cartoon-like images the GIF and PNG formats both reproduce the original information without artifacts. The only distinction of note between GIF and PNG is that GIF incorporates a patented compression algorithm.

        The JPEG format is ideal for photos and some scanned documents. Its compression method is lossy, but in most photos the loss of information is not readily detectable unless the compression is jacked pretty high.

        I don’t think Mr. Connolley has touched these files yet: http://en.wikipedia.org/wiki/Image_file_formats

        • Jonathan Schafer
          Posted Nov 5, 2009 at 11:28 PM | Permalink

          Re: Earle Williams (#6),

          Okay, I don’t want to hijack the thread into an image format discussion, but when you said

          For charts and cartoon-like images the GIF and PNG formats both reproduce the original information without artifacts. The only distinction of note between GIF and PNG is that GIF incorporates a patented compression algorithm.

          That’s not really accurate. GIF is an 8 bit image format. At 8 bits, it supports only 256 colors. The standardized web palette contains I believe 238 colors, so depending on how many colors you use, the image may appear different. If you have a chart with a gradient, or even a spaghetti graph that’s high resolution, saving as a GIF is going to produce artifacts in the resulting image. Converting a vector format to GIF can produce artifacts. PNG, OTOH, is a 24 bit image format with alpha-channel support. As a 24-bit image format, PNG supports up to 16.7 million colors. PNG is vastly superior to both GIF and JPG, and is of course, royalty free. PNG can be used for both photo’s as well as cartoon like images/charts. Conversion from vector images usually produces excellent replicas in bitmap format.

  2. Posted Nov 5, 2009 at 9:57 AM | Permalink

    Thanks again for calling the bluff on the excuse makers.

  3. Gary
    Posted Nov 5, 2009 at 12:16 PM | Permalink

    oops… my bad.

  4. Ron Cram
    Posted Nov 5, 2009 at 1:54 PM | Permalink

    Steve, good question! Where did the number 611 come from? Could it be a combination of Yamal and Polar Urals? 611-252=359. Were Polar Urals 359? Or is there another known series with 359? Or is 611 just a typo?

    I know a physical therapist who looked at a patient’s prescription for PT and said “The diagnosis is wrong.” The man heard the comment and became angry. “How can you possibly know that? You have not examined me! You haven’t asked me any questions!”

    The therapist replied “I could be wrong, but it says here you suffer from vaginal bleeding.”

    How does a mistake like that happen? The medical assistant typing the code into the computer simply entered the wrong code. Could that have happened here?

    • Steve McIntyre
      Posted Nov 5, 2009 at 2:30 PM | Permalink

      Re: Ron Cram (#7), There aren’t anywhere near 359 cores at Polar Urals.

  5. Posted Nov 5, 2009 at 2:46 PM | Permalink

    The 611 samples almost certainly include the samples that go back thousands of years before the beginning of the reconstruction. The samples that cover the period of the reconstruction number 252.

  6. Steve McIntyre
    Posted Nov 5, 2009 at 2:52 PM | Permalink

    The total number of samples back to the Holocene Optimum is 2171 plus 17. The sample size in Table 1 is said to be for the period -202 to 1996.

    We know something’s wrong with the table. It’s “possible” that 611 includes some samples dated prior to -202, but it’s possible that it’s a misprint or something that we haven’t thought about. It’s impossible IMO to be “almost certain” about what the explanation is.

  7. Posted Nov 5, 2009 at 2:58 PM | Permalink

    Fig. 3b of Briffa 08, at http://rstb.royalsocietypublishing.org/content/363/1501/2269.full.pdf, does show the Yamal counts as dropping off to 10 or so at the end, so that an alert world could arguably have known since 2008 not to give this series much credence. However, as Steve points out, no one could have known that in AR4 etc.

  8. Poha
    Posted Nov 5, 2009 at 3:19 PM | Permalink

    “If they don’t understand that notice in 2008 cannot be effective notice in 2000, it’s hard to have a sensible discussion. ”

    Typo; should read “cannot be effective notice in 2010 …”

    CA Rocks!

  9. Poha
    Posted Nov 5, 2009 at 3:24 PM | Permalink

    No typo, there: back to the future, for me. So obvious that it just did not scan for me, first pass!

  10. dearieme
    Posted Nov 6, 2009 at 4:26 AM | Permalink

    “If they don’t understand that notice in 2008 cannot be effective notice in 2000, it’s hard to have a sensible discussion.” I sometimes wonder if their aim is to “adjust” global average temperature until it exceeds their gormless average IQ.

  11. Alan Bates
    Posted Nov 6, 2009 at 4:32 AM | Permalink

    All this talk about image format … 8 or 24 bits, 1.7 million colours, or only 256 colours (for a black and white image), PNG, GIF, JPEG, OTOH (no, sorry, that was an abbreviation of something else).
    Personally, I think what was reproduced in Steve’s message is readable and has the stamp of authenticity (which is what this is all about). I am more impressed by a table that looks like it came from the original article than one that looks like it did not!

    I don’t come here to go Ooh & Ahh at the superior technical skill in blog presentation. Let’s stick with the point Steve is making.

  12. Dirk
    Posted Nov 6, 2009 at 7:55 AM | Permalink

    I don’t mind a bit of side-talk about formats as long as the info presented is accurate. It only took four posts to get to what looks to be an accurate presentation of web image superiority of .png format. If only climate science was that efficient.

  13. SteveF
    Posted Nov 10, 2009 at 11:52 AM | Permalink

    After DO’s initial idiotic reply about Briffa’s 2008 disclosure of core counts for the Briffa 2000 paper, I pressed him again:

    Me: “With regard to Briffa’s disclosure of the Yamal data set size for the 20th century, this was not (apparently) done in 2000, and 8 years seems to me a long time to not make this disclosure, especially considering the very frequent use of the Briffa 2000 reconstruction in other publications and in the IPCC analysis. I applaud Briffa for finally disclosing all the data that went into the 2000 reconstruction, but I wish he had done this years ago.”

    DO’s reply: “[1] Core counts were available in H+S2002, and if I understand correctly the data had been made available by request in 2004 (maybe Deep Climate can point me to where I read this). But in any case, I’m really not interested in second guessing what did or did not go into a manuscript nearly a decade ago.”

    DO knows the point is indefensible, so he has adopted the approach “I’m not interested in admitting anything, or even accepting reality, so I just want to move on.” I pressed him again.

    Me: “I think this is not a fair representation of what happened. Steve McIntyre has made very clear that at no time did Briffa explain that the H$S cores were exactly the same as used for Briffa 2000……”

    DO’s reply: “Your description is at odd with others, see http://scienceblogs.com/deltoid/2009/10/mcintyre_had_the_data_all_alon.php. I opened this door, kinda, so you had every right to walk through it, but no more on this, as it takes us away from the science.”

    Translation: “I will not ever admit that Briffa should have disclosed the low number of cores at Yamal, and I will not allow you to discuss it.”

    Too bad DO could not just have said right away: “Yes, Briffa should have warned about the low core count in his 2000 paper, since it reduces the confidence level for the last part of the reconstruction.” He would not have wasted my time and his, and not made himself look like an idiot.

    Like all other “climate scientists” that I have communicated with (or at least tried to), DO is incapable of normal discourse, because he is incapable of accepting even the simplest of errors. There is no point in trying to talk with such people, and it is foolish to believe anything they publish, since they are obviously not interested in the truth.

    • Steve McIntyre
      Posted Nov 10, 2009 at 12:12 PM | Permalink

      Re: SteveF (#18),

      As you say, climate scientists seem to be totally committed to never conceding even the smallest point – whether it be upside down Tiljander in Mann or Briffa’s non-disclosure of core counts. To me, it seems like a self-defeating tactic. Third parties can judge such issues for themselves and, when they see climate scientists stubbornly defending Upside Down Mann or non-disclosure of core counts, they draw conclusions about the scientists in question.

      Your characterization of my understanding of Yamal provenance is correct. I’ve learned to take care in trying to ensure that data set versions are the same – as they aren’t always. At no point did Briffa say that the Briffa 2000 data was the same version as the Hantemirov and Shiyatov 2002 data set. If that’s DO’s understanding, it is incorrect.

      Nor did I assume that they were the same version. Versions can differ and differ materially; for reasons discussed previously, I take care in trying to ensure that I’m working with the appropriate version. For example, the Taimyr version in Briffa et al 2008 turned out not to be the same as the Taimyr version in Briffa 2000. I may have been wrongfooted somewhat in respect to Yamal because Rob Wilson said that they had used Yamal because it was more “highly replicated” than Polar Urals – obviously not the case in the calibration period. Regardless of whether I should or should not have been able to deduce that the versions the authors of D’Arrigo were not aware of the low core counts at Yamal.

      I might add that the Taimyr data was important to the analysis as it showed Briffa’s inclusion of a Schweingruber data set, motivating the analysis of Yamal using Khadyta River. Regardless of what one may think of Yamal, the Taimyr data was a completely closed book prior to Sept 2009.

      • bender
        Posted Nov 10, 2009 at 7:37 PM | Permalink

        Re: Steve McIntyre (#19),
        Anybody tracking the story knows this. There’s gotta be, what, a dozen of us? 🙂 This DO character, like Connolley, is working with his eyes closed and his mind shut.

  14. steven mosher
    Posted Nov 10, 2009 at 12:55 PM | Permalink

    SteveF, DO has received his training in how to avoid questions and move on. I’ve got a fairly strong suspicion, backed up by a few facts, about his connections to RC. This all has to do with a media strategy of containment. Since I know about these things and have myself received training in these methods, let me assure you that scientists do not come up with these tactics on their own. The tactic amounts to this. You have a series of surrogates that do your attacking and defending for you. They have to be anonymous. Atmoz, for example, is NOT a RC surrogate. Tamino is a RC surrogate. He has published papers with RC folks. DO is also a RC surrogate. Connelly is also a surrogate. each surrogate specializes in different areas. The surrogates give RC and original authors several benefits. The surrogates can establish false lines
    of defense. you waste time and effort attacking their points. The surrogates ( say Tom P, who works in con junction with DO) can also try to waste your time and effort with bogus attacks. The other tactics they will use is one that drives most people crazy. They only let the LUNATICS from the anti AGW side post. Thus creating a false impression of the skeptics. ( Tom Fuler did a survey that showed that skeptics are NOT a monolithic group) They will drive you crazy by editing your posts, thus doubling your work as you cross post your posts to other other boards.

    More later. I’ll suggest some counter tactics.

    • Mark T
      Posted Nov 10, 2009 at 1:24 PM | Permalink

      Re: steven mosher (#20),

      The surrogates can establish false lines of defense. you waste time and effort attacking their points.

      Red Herrings are their specialty. Even on insignificant points posters such as DO put up multiple fallacies: red herrings, strawmen, ad hominems, always couched with some heavy hyperbole to point out how evil you are for not wanting to save the planet.

      The best tactic, IMO, is for people like Steve to ignore them.

      Mark

      • steven mosher
        Posted Nov 10, 2009 at 3:41 PM | Permalink

        Re: Mark T (#21), watch what steve does. He doesnt ignore them outright.

  15. curious
    Posted Nov 10, 2009 at 1:51 PM | Permalink

    Just because someone sets you a fool’s errand it doesn’t mean you have to go on it.

  16. clivere
    Posted Nov 11, 2009 at 7:31 AM | Permalink

    Re not conceding even minor points and the Tiljander issue in particular.

    My view based on the way events occured is that MBH08 did not understand the upside down issue at the time the original paper was submitted and still did not understand when they replied to the Steve and Ross response. I suspect that Mike Mann is not an avid reader of CA unlike someone like Gavin Schmidt who checks every comment with hand on phone ready for action. Gavin may not have communicated the error to him.

    Based on the recent publicity due to the Kaufman correction, William Connelley blundering into the fray without understanding the issue and the Finnish TV program I dont see any reason why they should no longer be aware. My expectation is they are obliged to issue a further correction which admits the error and probably leads to removal of the Tiljander proxies. However given the original response, the nature of the error and the publicity already received they are now in a real political bind about how they can do it because it will reflect badly on them whatever they do.

    Steve Mosher – I am not convinced by your assertions about surrogates and I dont think it is helpful for Steve to go down that road without strong evidence. I see a loose coalition of similar viewpoints with a level of interchange but plenty of differences. I am well aware that double standards operate in how they handle opponents vs supporters and DO is a good example of this. You would need to be more specific if you are going to assert conspiracy theories.

    • Steve McIntyre
      Posted Nov 11, 2009 at 7:56 AM | Permalink

      Re: clivere (#25),

      there is considerable contemporary evidence that they were monitoring the CA discussion on Mann et al 2008 and that Gavin and Mann were in coordination. A few instances: they promptly corrected their geographic mislocation of a Spanish precipitation series (the “rain in Spain”) in Kenya and re-calculated the effect a few days after this amusing error was reported here. Their first archive of “original” data wasn’t original data but RegEMed data – a point observed here. A few days later, they archived “original” data. Gavin was aware of Mann placing of the original data online within a few minutes of Mann actually placing it online – a point that was provable at the time. I happened to check at 11.30 am one day, no alteration had been made and I had a record of a 11.30 am check. Just after noon on the same day, by an inline comment, Gavin evidenced awareness of a change that had occurred after 11.30 am. – even though there was no change notice or other announcement of the alteration of Mann’s SI. They also modified their smoothing method after a problem was reported here – UC will recall the details. Although Mann and associates failed to acknowledge obtaining information from CA – thus somewhat distorting the research record for someone examining it now – it was evident that it wasn’t just Gavin monitoring the site or, in any event, he was in close communication with Mann at the time.

    • AMac
      Posted Nov 11, 2009 at 8:06 AM | Permalink

      Re: clivere (#25),

      > I don’t see any reason why [Mann et al] should no longer be aware [of the Tiljander proxy issues].

      Well. One would think that a bit of reflection on the issue might have preceded Mann et al’s Reply in PNAS to McIntyre’s & McKitrick’s Comment. It’s unusual (avoiding the ‘b’ word) that that doesn’t seem to have been the case.

      I wonder if the editorial board at PNAS thinks that things are okey-dokey. Their authors have twice reminded them that the journal’s formal mechanisms are irrelevant as far as Figure corrections. (The PNAS web page has only the original SI with the original Fig. S8a, at this writing.)

      Delayed Oscillator’s thread on Stoat/Tiljander is here.

      • bender
        Posted Nov 11, 2009 at 8:19 AM | Permalink

        Re: AMac (#27),

        Delayed Oscillator’s thread on Stoat/Tiljander is here.

        His understanding of the issue is worse than Connolley’s was. What do they want as proof that the proxy was used upside-down? A picture of a guy on a computer multiplying a column of data by -1? That ain’t the way it works.
        .
        Hint: Concede the point, re-do the graphs, thank the critics, and move on. Is it that hard to admit an error? Frightening.

    • bender
      Posted Nov 11, 2009 at 8:08 AM | Permalink

      Re: clivere (#25),

      William Connelley blundering into the fray without understanding the issue

      He “blundered into the fray” AFTER I explained to him, at Cruel Mistress, not just the issue but its cause. (He even credited me for such.) This insight gave him the power he needed to play authority at Stoat, where he would have editorial control. “Blunder” indeed! He tried to set a trap for “deniers” and “skeptics” – even warning them of the conditons under which they would be censored. He never thought he would encounter someone as rational and intelligent and persistent as AMac. AMac did not tell him anything he did not already know. He just showed Connolley for what he is – a propagandist looking to control the discussion.

    • steven mosher
      Posted Nov 11, 2009 at 2:54 PM | Permalink

      Re: clivere (#25),

      Re not conceding even minor points and the Tiljander issue in particular.
      My view based on the way events occured is that MBH08 did not understand the upside down issue at the time the original paper was submitted and still did not understand when they replied to the Steve and Ross response. I suspect that Mike Mann is not an avid reader of CA unlike someone like Gavin Schmidt who checks every comment with hand on phone ready for action. Gavin may not have communicated the error to him.

      mere speculation on your part. As steve points out we can be fairly confident that information on CA gets
      transmitted to Gavin and Mann. PLease the the mystery man episode.

      Based on the recent publicity due to the Kaufman correction, William Connelley blundering into the fray without understanding the issue and the Finnish TV program I dont see any reason why they should no longer be aware. My expectation is they are obliged to issue a further correction which admits the error and probably leads to removal of the Tiljander proxies. However given the original response, the nature of the error and the publicity already received they are now in a real political bind about how they can do it because it will reflect badly on them whatever they do.

      They will move on to the next study. They will rewrite the history. Don’t you get that by clipping and censoring posts
      THEY REWRITE HISTORY IN REAL TIME. Imagine this. I post something to WC. he reads my post. He sees that I have a point. He ammends his position. he blocks my post. he takes credit for my insight. I don’t think you fully understand the media strategies involved here and the organization that developes and refines them. You go research the
      organization that hosts the RC blog. Then you get back to me.

      Steve Mosher – I am not convinced by your assertions about surrogates and I dont think it is helpful for Steve to go down that road without strong evidence.

      Hmm, but look YOU had no problem speculating on NO EVIDENCE that gavin doesnt read this blog, when it’s clear he does. Same for Mann. You had NO PROBLEM speculating on NO EVIDENCE when the goal is to clear the team of wrong doing. BUT, when I suggest connections ( based on evidence) you have a problem with that. Hmm, that assymetry in your epistemology implies a bias, or ignorance. you choose.

      I see a loose coalition of similar viewpoints with a level of interchange but plenty of differences. I am well aware that double standards operate in how they handle opponents vs supporters and DO is a good example of this. You would need to be more specific if you are going to assert conspiracy theories.

      Of course there are differences. But differences that make no difference make no difference.
      When I make a case about the SIMILARITIES and you point out differences, you are just being obtuse. You obviously haven’t been around the blogs long enough to see the patterns. If you want solid evidence of course you realize that I would be violating their anonymity. In the case of Tamino I had no issues outing him with others. he accused sceptics of being criminals. So, we said his name. Same with rabbit. He went off on a 15 year girl, so we outed him. WRT Tom P and DO and the other surrogates… They haven’t crossed the line yet.

      snip – prohibited word

      So, I’m entirely happy saying that the surrogates have learned their role and evolved into their role as an adaptive strategy. Its a strategy nonetheless and I don’t really care if there is an orchestrated effort. its a difference that makes no difference.

      Finally, steve doesn’t go down the conspiracy line. But he is aware of the various team tactics and strategies, in fact he is a great cataloger of them, as is bender.

  17. clivere
    Posted Nov 11, 2009 at 9:18 AM | Permalink

    Bender – I agree that your post at CM was the first point that William started to recognise that the issue was genuine but I believe he was first drawn into the fray by an intentionally provocative post by RPjr.

    I think we all agree that the error is a very silly one. I personally believe it is quite a serious one that illustrates a number of issues with these types of reconstructions and is also the basis of a strong justification for someone like Steve to continue to review these studies. Other types of review dont appear to be working!

    Steve/AMac – there is an old saying that the cover up is worse than the crime. I still think it likely that Mike Mann did not understand and was not fully aware. If not then he is compounding his error by trying to bluff it through. To me their only viable tactic from a political perspective is to recognise the error but pursue the “it does not really matter” argument. They will only be able to get away with that if the error, its magnitude and its impact is not properly documented. I would still urge Steve to produce a concise journal quality post. The earlier posts were not written with the view to this issue becoming a major battleground.

    • bender
      Posted Nov 11, 2009 at 9:41 AM | Permalink

      Re: clivere (#30),
      Some people are being willfully obtuse when it comes to the points Steve M is making. Connolley and Rabett are prime examples: willing to misrepresent what Steve said and did – blindly defending the guity – until I track them down and give them their deserved smacking at CM. Whereupon they retreat to their religiospheric echo chamber.
      .
      Hint: when Steve M makes a point, you may want to investigate before leaping to conclusions.

  18. steven mosher
    Posted Nov 11, 2009 at 3:00 PM | Permalink

    steven mosher (#32), arrg.. we need the edit function here. Correcting the flame above. hehe.

    Hmm, but look YOU had no problem speculating on NO EVIDENCE that gavin doesnt read this blog, when it’s clear he does. Same for Mann. You had NO PROBLEM speculating on NO EVIDENCE when the goal is to clear the team of wrong doing. BUT, when I suggest connections ( based on evidence) you have a problem with that. Hmm, that assymetry in your epistemology implies a bias, or ignorance. you choose.

    Hmm, but look YOU had no problem speculating on NO EVIDENCE that gavin didnt transmit information to Mann, when it’s clear he does. You had NO PROBLEM speculating on NO EVIDENCE when the goal is to clear the team. BUT, when I suggest connections ( based on evidence) you have a problem with that. Hmm, that assymetry in your epistemology implies a bias, or ignorance. you choose.

  19. clivere
    Posted Nov 11, 2009 at 6:30 PM | Permalink

    Steve Mosher – I have followed both RC and CA from the outset. I am an observer rather than a player with just a single post at Stoat and approx 30 posts here and at Lucias. I dont think your conspiracy speculations are helpful to Steve regarding the public image of the quality of discussion at this blog but we can just disagree on that. It is Steve’s blog and his call to make. I have no further interest in discussing it.

    With regards to whether Mike Mann understood the issue I would pose the following questions

    1. Given that they corrected other errors when identified why did they not wish to correct this one?

    2. Did this error have such importance to them that they had to bluff it through? Why make this issue a major battleground?

    3. How did they think they could win the argument by not admitting to it?

    4. Does the MBH08 response show a clear appreciation of the issue or is it only responsive to a partial understanding?

    5. Given space limitations does the formal letter by Steve and Ross document all the details such that the mechanism that caused the inversion is clear?

    6. The place where the issue was documented at CA at the time was part way through the Saturday Night Live thread. If someone was not alert to the issue at the time of the posting how easily would they find those details at a later date? For example does the thread title clearly define the details contained within?

    • Steve McIntyre
      Posted Nov 11, 2009 at 7:09 PM | Permalink

      Re: clivere (#35),

      Blog rules prohibit talk of “conspiracy” as pointless and counterproductive. Having said that, there’s no reason why realclimate coauthors Mann and Schmidt shouldn’t be in touch with one another and in coordination; there’s evidence that they were during our discussion of Mann 2008 and surely one can make such an observation without this being conflated with “conspiracy”.

      1. It seems to me that their present preferred style of defence is to treat any observation of an error as nothing more than a typo. This was their approach in the MBH98 Corrigendum. In the Corrigendum, they prominently fessed up to making an incorrect citation as though that were representative of the problems, while not mentioning the principal components problem or various other issues. (Ironically, their revised citation remained incorrect even in the Corrigendum, but this is the Team, after all.) This sort of defence has played well among climate scientists, who, for the most part, seem to have bought into the idea that all the errors, large and small, do not amount to more than typos. Thus, they will correct errors that they know don’t matter or which can be construed as typos.

      2. Bluffing through things has , for the most part, worked for them. Think of the verification r2 thing.

      3. Past experience. Mann admitted nothing in respect to MBH.

      4. One never knows. The issue is trivially simple. Dozens of CA readers who are not climate scientists have understood the error effortlessly. It’s only climate scientists that seem to have “trouble” understanding the error – which makes one think that they are being wilfully obtuse. But perhaps Mann and associates didn;t understand what CA readers understood easily.

      5. The space limitations prevent anything more than listing issues. If matters were reversed and I were trying to understood an issue listed in a Comment that wasn’t entirely clear to me, I would look closely at related posts at realclimate.

      6. There is a left frame Category (Mann 2008) where the post can be readily located (and it was even easier at the time.) CA threads turn up in google searches as well.

      • bender
        Posted Nov 11, 2009 at 9:56 PM | Permalink

        Re: Steve McIntyre (#36),

        they prominently fessed up to making an incorrect citation as though that were representative of the problems, while not mentioning the principal components problem or various other issues

        -give ground, appear reasonable, but don’t give in
        -asinine, typical

      • Posted Nov 12, 2009 at 3:11 AM | Permalink

        Re: Steve McIntyre (#36),

        Blog rules prohibit talk of “conspiracy” as pointless and counterproductive.

        Or rather, that’s what they want you to believe…

        • Dave Dardinger
          Posted Nov 12, 2009 at 7:56 AM | Permalink

          Re: John A (#38),

          Or rather, that’s what they want you to believe…

          From your blog:

          Climate Audit is one such blog (that I’ve been proud to have been a small part of)

          So should’t it be:

          Or rather, that’s what we want you to believe…

          [OK, I’m just nitpicking. I know you were just being funny.]

      • Mike B
        Posted Nov 12, 2009 at 7:35 AM | Permalink

        Re: Steve McIntyre (#36),

        One never knows. The issue is trivially simple. Dozens of CA readers who are not climate scientists have understood the error effortlessly. It’s only climate scientists that seem to have “trouble” understanding the error – which makes one think that they are being wilfully obtuse. But perhaps Mann and associates didn;t understand what CA readers understood easily.

        I’d say it’s a matter that Mann et. al. didn’t even try to understand.

        Among those who openly defended him, it was a combination of “The Great Mike Mann can’t possibly be wrong” coupled with a only a superficial understanding of statistics by many paleoclimatologists.

        And there were probably some paleos who understood the error, but were reluctant to speak up out of fear or indifference.

    • EW
      Posted Nov 26, 2009 at 3:47 AM | Permalink

      Re: clivere (#35),
      clivere, the Tiljander data problem lies not only in the upside-down use but also in the fact, that the hockey-stick part of these data is influenced by man-made varve disturbances in the proximity of the lake in 20th century. Which the authors clearly stated in the original paper.

      • AMac
        Posted Nov 26, 2009 at 9:52 AM | Permalink

        Re: EW (#48) & clivere (#49)

        Re: the Tiljander (Lake Korttajarvi) varve proxies. A reasonably-well-infomed reader (e.g. me) isn’t able to figure out how Mann et al (PNAS, 2008) used the ‘screened’ long-term proxies (pictured in Fig. S9). The procedures involved in the assembly of the paleotemperature reconstructions (e.g. Fig. S8a) are only opaquely described in the Supplemental Information. I’m not talking about the specifics of the statistical approaches, which I wouldn’t expect to appreciate without a good deal of background reading. I mean general conceptual approaches such as weighting, and their implementations.

        During the back-and-forth at Stoat and lately at RealClimate, it became clear that even the best-informed of the many ardent defenders of Mann et al (2008) don’t understand these issues any better than I do–at best. It’s been mostly a matter of cheering the home team, with a bit of hubris added to the mix. It’s not evident that PNAS’ peer-reviewers and editors had any insight into the severe and fairly obvious methodological issues that plague this aspect of Mann et al. Perhaps they lacked interest.

        Gavin Schmidt’s recent justification in the comments at RealClimate of Mann et al’s use of the proxies is copied at the Air Vent.

        • bender
          Posted Nov 26, 2009 at 10:15 AM | Permalink

          Re: AMac (#50),
          And that, AMac, is exactly why so many of us have soured on anything to do with “The Team”. The pattern of incompetence and denial and revisionism is pretty robust.

  20. Vangel
    Posted Nov 12, 2009 at 8:33 PM | Permalink

    I was wondering if there are any peer reviewed papers that use the same type of methodologies and Mann, Briffa, Esper, etc., and come to a conclusion that the proxies show no hockey stick shape. Can anyone point to a paper that includes the full Yamal set, Plar Urals, etc., and shows that there is no major warming trend present? If not, why isn’t the work being done and sent out for publication.

    • bender
      Posted Nov 12, 2009 at 9:00 PM | Permalink

      Re: Vangel (#41),
      I will write the paper you request on the condition you tell me just one thing. How do you reconstruct one parameter – temperature – when growth is co-determined, nonlinearly, by the interaction between temperature and precipitation? i.e. drought.
      .
      Supposing you can’t do this (hint: you can’t), then how do you go about honestly representing the model mis-specification error in your weak approximation? If you don’t udnerstand what I’m saying, then read the blog. In particular my crosspost of yours in the ‘Wilson on bristlecones’ thread.

  21. Chris W.
    Posted Nov 24, 2009 at 5:58 AM | Permalink

    Apparently the 611 came from yamal.rwm, which was sent to Briffa from Rashit Hantemirov in Oct of 98. The file is in the yamal directory in the leaked files, the explanation is in an email from Rashit (Just search for yama.rwm in the leaked emails, it will be the only occurrence.)

    Seems like Briffa’s claim of having done-all-the-work-so-why-should-I-show-it-to-you is completely BS. All of the painstaking work of chroning the 611 trees was done by the Russians and sent to him as a single file.

  22. Chris W.
    Posted Nov 24, 2009 at 7:23 PM | Permalink

    BUSTED! I’ve been looking through the data and it appears as if only 257 of the 611 samples cited actually contained any post 200B.C. data.. Whoops!

  23. RomanM
    Posted Nov 25, 2009 at 7:08 AM | Permalink

    The rwm format was a little different from the previous ring files we have seen here lately. I presume that Steve might already have a script to read them, but I like doing my own. If anyone wants to play with the data, they can read it using the following:

    # fil = filename of rwm file expected to be in default directory
    # result is in the same format as Steve was using in the earlier yamal work:
    # gives data frame with four variables: is, age,year, rw

    read.rwm = function(fil) {
    datlin = readLines(fil)
    dn = length(datlin)
    id = NULL
    age = NULL
    year = NULL
    rw = NULL
    #read data
    lin = 1
    beg.seq = 1+3*(0:25)
    end.seq = beg.seq + 2
    while (lin < dn) {
    cor.age = as.numeric(substr(datlin[lin],5,8))
    styr = as.numeric(substr(datlin[lin],14,18))
    cor.id = substr(datlin[lin],22,27)
    lin = lin+1
    rws = rep(NA,cor.age)
    nlin = floor((25+cor.age)/26)
    linlens = c(rep(26,nlin-1),cor.age-26*(nlin-1));k=1
    for (i in 1:nlin) {beg = beg.seq[1:linlens[i]]; en = end.seq[1:linlens[i]]
    for (j in 1:linlens[i]) {rws[k] = as.numeric(substr(datlin[lin],beg[j],en[j]));k=k+1}
    lin = lin+1}
    id = c(id,rep(cor.id,cor.age))
    age = c(age,1:cor.age)
    year = c(year,styr:(styr+cor.age-1))
    rw = c(rw,rws)}
    data.frame(id,age,year,rw)}

    #Example useage:
    #newyam = read.rwm(“yamal.rwm”)

    This script expects the file to have no extraneous lines in it. I have a second script which allows for that and also produces some extra descriptive information about the data (id, core age, start year and end year).

  24. Chris W.
    Posted Nov 25, 2009 at 12:58 PM | Permalink

    Ispent yesterday writing a matlab script that extracts and can test plot the data.. I will try to get it up later today. Here is the raw Yamal plot for the last 2000 years with a 21 year rolling average in red. I have to fix the encoding on the image later, as it ended up being jpg while I wanted png.

    Chris

  25. clivere
    Posted Nov 26, 2009 at 8:10 AM | Permalink

    EW – not sure what has triggered your comment. I did try to provide my understanding of the issue in the Rank Gavin Noise thread

    http://www.climateaudit.org/?p=7619#comment-364239

    I have largely been watching this from a distance and only decided to get involved when I thought Steve was in danger of appearing to lose the argument due to the oppositions ignorance of the details and mechanism plus Steve’s own belief they were just doing it all to annoy him. Steve wrote his original posts for his own CA audience and whilst they explained the issues well enough at the time it was not expected to be a battleground. The more recent posts were too preoccupied with snarking at the oppositions ignorance rather than clarifying / explaining why the issues were significant.

    Whilst we know from previous experience and Climategate that they are quite prepared to act in ways that will annoy him I dont believe they would do this from what is likely to be a losing position. I still believe it is more plausible that the original reply by Mann and William Cs early involvement were through ignorance rather than malice. I am waiting to see if they will eventually own up or will continue to bluff. Gavin Schmidt was doing some bluffing in a recent response to AMAC at RC.

    The interaction between several blogs has been a great source of interest and entertainment for me. I have been following the timeline of events quite closely and as far as I can establish early September this year was the first time that awareness of the error was acknowledged by Kaufmans group. Mike Mann was only copied on the 3rd of the Kaufman emails exposed by Climategate. I have also seen a comment by Mike Mann in one of the emails that he rarely reads CA so would have been reliant on someone like Gavin notifying him. This issue turned up a few weeks after the original paper so Gavin may have missed it or not understood the significance. In many respects it seems a rather unlikely battleground issue so may not have been passed on.

    I still remain of the view that the error and its impact is not properly documented anywhere in a manner that is accessable to the impartial outsider. Gavin Schmidt will direct people to Stoat where it is not adequatly described despite the valient efforts of AMAC. If someone then comes looking at CA they may struggle to find it amongst the huge number of threads. They may not recognise the Saturday Night Live thread as relevent. Possibly they may recognise the Upside Down Mann thread but in Steve’s shoes I wouldn’t have left it to chance. If you go to Stoat and other blogs such as Grumbine you will still see people in ignorance insisting that the proxies were not used upside down or the issue does not matter.

    However this is now all overtaken by and is a much lower priority than the multiple issues raised by Climategate and where amonngst other things we can observe real evidence of collusion. I expect Steve is suffering from information overload at present and this issue should now be very low down his list.

  26. Chris W.
    Posted Nov 28, 2009 at 12:41 AM | Permalink

    I meant to have this up the night before last, but Thanksgiving happened.. (And Happy Thanksgiving to all!)

    Here is a matlab file that reads the .rwm files, with descriptions for what it gathers from it.
    According to sf2.txt in the leaked files, the units are .01 mm.

    http://pastebin.com/f77433b10

    Here is another matlab file that plots the data that the first function returns. It isn’t meant to signify anything in itself, It’s mainly for example.

    http://pastebin.com/f6694e01e

    Here is the (plotted) output for the entire 611 core Yamal history…

    Happy hunting,

    Chris