Nature reported today on the CRU data requests. I was interviewed at length last Thurs, followup Friday by Olive Heffernan of Nature. They even asked for a photograph. I haven’t seen the article yet. More after I see the story.
Update: There is an additional discussion at the Nature Blog. Behave nicely.
Update- my picture is in the article :). Photo credit is to my sister.
Climate Audit 
68 Comments
Growing demands for access to information swamp scientist.
by Olive Heffernan
Yeah, copying the data to an FTP directory is incredibly onerous.
Surely not if all data is just put up on a server for people to access as and when they please? If a particular subset is required then surely any researcher would be capable of extracting it?
No mention of CRU’s use of GHCN data, no mention of the public FTP site that had the data for years. Just a poor old climate scientist being harassed by a bunch of wacky skeptics.
Can’t anybody in the climate field get anything right?
Jones has not “been swamped by a sudden surge in demands for data.” Jones said that he was unable to provide data because of existing confidentiality agreements. People asked for documents showing the existence of the confidentiality agreements. Jones failed to do so.
I wouldn’t say that I was “especially aggrieved” at Peter Webster getting the data. Peter Webster should get the data. Peter Webster has been very cordial to me and I regret that his name gets mentioned in the same breath as CRU stonewallers.
“Aggrieved” implies that I’m emotional about this. I’m not.
My point is entirely procedural. CRU provided data to a “third party”. Procedurally this showed that they were entitled to do so under their agreements and legislation. Then the question is whether the exact language of their agreements established a distinction between a request from Peter Webster and from me.
Their “agreements”, such as they are, clearly don’t.
Re: Steve McIntyre (#5), I am still waiting for a reply to my request for “a copy of any digital version of the CRUTEM station data set that has been sent from CRU to Peter Webster and/or any other person at Georgia Tech between January 1, 2007 and Jun 25, 2009”; they still have about a week of their 20 working days to run.
Re: Steve McIntyre (#5), I’m geting confused. I thought they said the data was all lost.
Steve: Just non-“value added” data – it’s like having their GISS dset1 version in which versions have been combined.
Re: PhilH (#19), I am still confused. What value is the “value added” data if you don’t know what raw data they started with or how they added the “value.”
Steve- you’re not the only one who’s confused about this.
Re: Steve McIntyre (#5),
Mr McIntyre,
If Peter Webster is such a cordial chap. Why not get the data from him?
Or is he perhaps hampered by agreements not to distribute, just as CRU is?
In which case try the old foi trick
Re: thefordprefect (#40), Not a bad suggestion about asking Peter Webster for the data he has. In my FOI I asked for any instructions or stipulations accompanying the transmission of data to Georgia Tech. The answer I got today was
Ironically, in the same letter in which they refused to allow further transmission of the data to me that they had transmitted to Georgia Tech, they referred, inter alia, to the following grounds:
There is no mention that they can’t release the data to a non-academic, since of course that does not apply to me. That was always a red herring. They just make up reasons for the immediate purpose, even if they contradict reasons they made up 10 minutes earlier.
Re: Ross McKitrick (#53),
That’s ok then. You have your data! I’m sure Mr. Webster will be pleased to give you what you desire with no hassle!
Re: thefordprefect (#56),
It is totally unreasonable to involve Peter Webster, who doesn;t deserve to have his name mentioned in the same breath as the CRU Gong Show. I hope that no Climate Audit readers bother him and will communicate this to Ross.
In his shoes, I would certainly refer any request back to CRU, where it belongs.
There are a variety of appeal possibilities available for each of the refusals by CRU and, who knows, perhaps some CA readers will appeal the CRU decisions.
Re: Ross McKitrick (#53),
PS I hope you will be making the data available in the database here for all to use?
I see set the dogs loose on Nature, great.
Re: thefordprefect (#57),
Now why would you want to do that, TFD?
In any case, I see only one mutt there, and he’s toothless anyway. And incontinent. Wouldn’t your kids let you put ol’ Fella bigcitylib down? It would be a mercy if you did.
Congratulations Steve on getting this in print.
Dr. Jones did a lot of backpedaling I see, a bit like a unicycle rider on a ski slope.
And in all cases there is absolutely no physical evidence that any agreements exist.
I realize that you said Peter Webster was helpful but this statement doesn’t make any sense when the data should be freely accessible on a ftp server – as it apparently the 03 data was.
Not to imply that Peter Webster was in any way referring to CA but those who are familiar with CA (including Dr. Jones) know that the data would be used ‘authentically’ here. That’s IMHO the crux of the problem.
The story is also up at Climate Feedback, including apparently some details that didn’t make it into the Nature article. You can also comment without having access behind the paywall.
Inexplicably, Olive reported Jones’s excuses for withholding data without mentioning his most famous:
Climate Feedback includes the interesting information that Dr Jones has challenged Steve to produce a global temperature record once the data is available.
Re: Bishop Hill (#10),
Nice innuendo there:
Any thoughts on how much money Dr. Jones has made over the years peddling his shoddily maintained index?
Congratulations. The ball is moving.
The Nature article however very subtlely makes you look like a bit of a trouble-maker, and that pesky you are impairing their important work:
What irks me is their line:
They know full well who you are, your capabilities, and how serious you are with these matters. And Jeff Id is right on the money in reminding us they have yet to produce any hard physical evidence of agreements. They’re slipping, squirming and twisting. C’mon, just open up lads!
So how many NDAs did the dog eat?
Seriously, verbal and misplaced NDAs? Who is he trying to kid?
I submitted the following at the Nature blog:
Now that people asked for confidentiality statements and have read the various statutes, we know that CRU is required to ask the countried as requried by FOI, right? It looks like the data may become available.
Re: lucia (#14), yes correct. Annex G as I pointed out requires them to ask third parties about disclosure. That correspondence will then become the object of subsequent FOIA requests unless they post those agreements.
I don’t think that Jones is the right guy to organizing a set of permissions. It needs someone fresh.
Hey folks, my picture’s in Nature 🙂
Re: Steve McIntyre (#18),
Hey, you are not bad looking…. well for a guy…. and a Canadian.:)
Somehow the phrase, “take the lead”, seems like an overstatement…
Hate to nitpick…. BUT, …. Nature and Jones means the agreements were made “orally.” All agreements are verbal except perhaps those made with a wink and a nod.
Nice parthian arrow from the Nature journalist here… Steve you are single handedly responsible for making data availability more difficult… Congratulations.
Re: Antonio San (#22),
climate scientists often have an interesting approach to cause-and-effect.
Let’s see: as a result of the FOI requests in July 2009, Phil Jones had already commenced becoming less responsive a few years ago. Hmmm,…, maybe it’s a positive feedback.
I’m trying to figure how much less responsive his present refusal was then his refusal of Warwick Hughes or Willis Eschenbach.
Re: Antonio San (#23),
Except that the reality is that the data was previously refused for various and sundry reasons, but the Met office now claims to be happy to make it available and Jones is evidently beavering away to eliminate the impediments to disseminating it. (A task Jones did not previously find the time to undertake.) 🙂
Re: lucia (#32),
I can imagine jones’ letter.
Dear NMS,
The forces of evil skeptics are demanding that we release the data you so graciously supplied us. We have
fought for over 7 years to fulfill our contractual obligations to preserve your data and supply it only to those
legitimate climate scientists who endeavor to push the science forward. At this time however we are under assault
and hordes of amateurs are requesting access to your data. These untrained people will wreck havoc with the consensus we have so painstakingly labored to establish. There only purpose in requesting this data is to find something wrong with it and then demand that nothing be done about global warming. Therefore, we ask you to do your part. We believe that this climate data should be made freely available to all legitimate climate scientists for legitimate climate studies. We, are therefore recommend that you send us a contract to govern the use and distribution of your data. We would suggest an agreement that made the following points:
1. Support for data transparency and public availability of climate data.
2. Appropriate distribution restrictions that make this data available to legitimate climate scientists. We would
suggest that we can work with you to decide who is and who is not a legitimate scientist.
3. An appropriate review process for proposed studies using your data.
By memorilaizing these three elements we believe that we can both make the data available and prevent it from falling into the hands of deniers. If you have not entered into an agreement with us, then now is the time to do so. This is your last chance to be part of the famous CRUTEMP global index.
Best Regards,
Phil Jones.
PS. I’m looking for side work in data management and document control. Let me know if you have any work. I’ll
give a co authorship position to someone on your staff.
The idea that you must certify that you are an academic to get the data is ludicrous. BUT the idea that you must agree to co-publish with Jones to get the data–well, that is all very academic. They do not grasp that this is NOT academic world business as usual anymore.
Steve, all your efforts are much appreciated.
I sincerely hope the non-value-added data gets released to the public sometime in the next year (a simple ftp site would do me).
I have to recant – no “lying in the weeds”, “sitting on their hands”, or “masterly inactivity” (Humphrey Appleby) 🙂
When I read something in print, even in an ostensible science outlet, I don’t assume that it’s being written by a scientist. Many (most?) are journalists. The decline in the professionalism of journalists in recent decades is a separate issue, but I’d say that it’s more likely that that’s what’s really going on here.
As long as you have your hot dog, and you like the taste, why do you care what animals and organs are in there?
Re: Calvin Ball (#30),
Some people want to know if things are kosher in the figurative sense of the word.
I am not happy to see CoolChill post Nature’s pay-per-view article here.
Since the CA community is so enamored of ethical behavior most of the time.
Lucia, of course! I was being facetious in #23… next time I’ll put a 😉
my comment on nature
You wrote:
“Jones says that he tried to help when he first received data requests from McIntyre back in 2002, but says that he soon became inundated with requests that he could not fulfill, or that he did not have the time to respond to. He says that, in some cases, he simply couldn’t hand over entire data sets because of long-standing confidentiality agreements with other nations that restrict their use.”
1. Jones had no problem supplying the data to peter webster, so his reason was not FOI fatigue.
2. the 58 requests were not for data. they were requests for the AGREEMENTS.
You wrote:
“Given that McIntyre’s wish for access to the data will take time to be granted, this dispute will likely continue for some time. He’s especially aggrieved by the fact that hurricane expert Peter Webster at Georgia Tech University was recently provided with data that had been refused to him. McIntyre’s point here is that he should be treated as a legitimate academic given his background and publication record.
But Webster points out that he was allowed access because of the nature of his request, which was very specific and will result in a joint publication with Phil Jones. “Reasonable requests should be fulfilled because making data available advances science”, says Webster, “but it has to be an authentic request because otherwise you’d be swamped”. ”
1. Steve is not the least bit emotional about the webster access to the data. (personal communication)
2. None of the PURPORTED confidentiality agreements set preconditions that webster lays out.
3. Posting the data on a FTP server obviously did not swamp CRU when they did so accidently.
Re: steven mosher (#36),
1. Jones has said he will work to free the data.
2. Until the commercial data has been freed up sending data to a blogger not an academic would seem a bit hairy. Could McIntyre be trusted to keep the dta to himself or would it get dispursed amongst many.
3. Until the data is free it cannot be posted on an FTP site. Individual application have to be assessed.
4.see thefordprefect (#40). Can you extract the data from this source?
Re: jlc (#38), You have every right to go out and purchase the data if it is commercial from the sources. You have no right to ask for the data for free from another who has purchased it.
Re: Steve McIntyre (#25),
From what you said on previous threads I thought that the requested data was “accidentaly” posted on the FTP server after a request of someones. It would seem to have been forgotten and left undeleted for too long, but it still looks as if CRU tried to help in an underhand sort of way.
Re: thefordprefect (#43),
So you wish us to believe that in recent years Jones has actually been trying to help Steve gain access to the data — and that he wanted to help so badly that he violated all those confidentiality agreements by leaving the data on an FTP server?
I don’t believe that even you can believe such a thing.
Off topic
The concept of making data, of whatever nature, available only to “academics” is obscene.
As an engineer working in climate related fields for 40 years and actually using data produced by academics, engineers, hydrologists, meteorologists, sedimentologists, vulcanologists, and people paid a pittance to collect, compile and process data (not always well) throughout the developing nations,I believe I and my colleages have some rights here.
What is the point of collecting this data? It is not an academic exercise. We use climate data every day. Erroneous data can lead to wrong decisions that could have many repercussions.
Well done Steve, well done!
Reply to OT
Congratulations Steve! Here is my contribution to Nature’s blog:
Tim Davis, I can point you in the right direction. And yes, it is astonishing and appalling. Go to http://www.climateaudit.org/?p=559 and you can read Steve McIntyre’s report which contains a link to a PowerPoint by Hans von Storch who confirmed this exchange with Phil Jones.
Funny. As in Funny Strange. It’s natural for some blogs to have a time delay between when a commented is submitted and when it is posted. My comment was sent to Nature’s blog prior to Bishop Hill’s comment being posted. We are basically saying the same thing except I provided a link to CA which links to Hans von Storch’s PowerPoint. Now I am seeing comments posted which refer to a comment by bigcitylib posted after my comment was submitted. It is a little annoying and makes me wonder if Nature is not going to provide the evidence I gave to back up the statement Bishop Hill and I both made.
Re: Ron Cram (#45),
I was looking out for your post too. It certainly seems it may have been “lost”. Tsk tsk! What awfully bad luck. Perhaps a high-energy cosmic ray zapped your post. It must be something like that as I’m sure there’s no skullduggery going on at Nature. No, surely not at Nature.
I think you should resubmit it.
Re: Ron Cram (#46),
Why would the Nature editors not permit a link to the CA thread? Strange.
For some reason, I thought that I saw your comment up at the Nature blog this morning, but maybe I saw it here.
Re: Steve McIntyre (#48),
Bishop Hill (August 13, 2009 11:09 AM) mentions Hans von Storch but doesn’t provide a link. It was only the absence of the link that alerted me to the fact that it wasn’t Ron Cram’s post, which I assumed it was. At first.
So, I guess it was a-man-in-a-hurry thing for you, maybe?
Re: Ron Cram (#46), Ron:
I tried a second response at Nature and they bumped it with a message saying they were restricting responses ostensibly to control “tone”. I tried to politely respond to bigcitylibs misleading and gratuitous post.
I’ve tried to post another comment with the link, just in case. IIRC the moderators clock off at 5pm at Nature so it may be tomorrow before it gets through.
Is it true that Jones has also not described his algorithm in a public forum?
Well I’m an old rock slinger, with maths that linger,
Seems I’m reviled everywhere I’ve gone.
Cause I blog about duty and I blog about truth,
With every two cents I thrown.
I’ll scrape any number set, and blog results that I get
But the datum that I’ve never known
Is the points that’ll get you when you get your picture
In an issue of the Nature News
{Refrain}
Nature News – Wanna see my picture on the cover
News – Gonna buy five copies for my mother
News – Wanna see my smilin’ face
In an issue of the Nature news.
I’ve got a freaky old friend name of Roman M
Who embroiders on my graphs
I’ve got poor “old” Hu McCulloch
Checkin out all my maths
Now it takes all kinds, of intelligent minds
But these minds won’t really be shown
Like the show that’ll get you when you get your picture
In an issue of the Nature News
{Refrain}
I’ve got a lot of smart math groupies
Who’ll check out anything that I say
And some highly educated professors
Who’ll help out without any pay
I got all the friends that blogger could want
So I never have to blog alone
Now we’re showing our trews and spreading our views
In an issue of the Nature News
{Refrain}
I’m glad that this got reported in Nature. Now we’ve got to see if it does any good…
Steve McIntyre:
Hey folks, my picture’s in Nature
Wasn’t there a pro wrestler who went by the name, “Nature Boy”? Seems like there’s another one now. 🙂
In blog comments over at Nature, Olive Heffernan appears to be claiming that Mann reproduced his hockey-stick results fairly well by discarding the bristlecone series, which appears to be fairly at odds with McIntyre’s work, unless I’ve completely misunderstood.
Which is always possible.
Re: Slartibartfast (#62),
I’ve posted a comment over there on that claim.
Thanks for that, Steve.
At least they’re letting Steve post in his defense. I tried posting just now and got this message:
I submitted two comments to two different posts last weekend that finally got published on Tuesday. I guess three in one week have fingered me as an “abusive user”. We’ll see if BCL gets another one published.
Anyway, for the record, here’s my rejected post to the Nature thread on Steve Mc.
Re: John M (#65),
I got the same message today at about 16:00 GMT (approx 5pm London Summer Knocking-off Time). My previous post had been about 15 hours prior to that. I closed and ‘flushed’ my browser in case it was just some simple glitch, waited 10 minutes and tried again. It worked, or seemed to, but the post has yet to be put up. It was similar yours in some respects. It wouldn’t hurt to try again.
.
BTW given the topic, I’m a little surprised Nature doesn’t appear to have received more posts on it.
Re: John Archer (#66),
50 comments on one thread is equivalent to an avalanche for the Nature blog. They’ve been in operation for more than two years and only have a total of 1145 comments.
They had a couple posts with 30+ comments in May 2007 when they started. Their slant on what was acceptable for discussion was made clear very early and people left in droves.
Nature Blog Withdraws Invitation
Robustly homogenized temperature. I’ll be more interested to reproduce Brohan’s uncertainties once the data is available. Slightly on topic, there was interesting article in IEEE Xplore Top 100,
Reproducible research in signal processing
Vandewalle, P.; Kovacevic, J.; Vetterli, M.
Signal Processing Magazine, IEEE
Volume 26, Issue 3, Date: May 2009, Pages: 37-47
(see also http://www.reproducibleresearch.net/)
MBH98 seems to be of degree 2 (extreme effort of few years), MBH99’s grade is clearly 0 until script to reproduce CIs is available.
Internet test for transmission
3 Trackbacks
[…] health care. What is wrong with everyone? You can sense the anger, the resentment. And the panic.”Nature Reports on CRU Stonewalling « Climate AuditNature reported today on the CRU data requests. I was interviewed at length last Thurs, followup […]
[…] Another UK climate data scandal is emerging 14 08 2009 As many WUWT readers know, Steve McIntyre’s tireless quest to get the raw data that makes up the gridded Hadley Climate Research Unit has been fraught with delays, FOI denials, and obvious obfuscation. […]
[…] She didn’t report that it in her story, but I added the comment at the Nature blog and CA at the time (Aug 12, 2009) as follows: Ross McKitrick has long observed that important indices like […]