I’ve set up a new category at right entitled "Archiving" which cross-classifies many of the posts on data archiving and requests. A review from this time last year is here . Since then, I’ve had correspondence with Moberg and Nature, which has resulted in winkling out the other Moberg series (which I’ve yet to process); extensive correspondence with Science about Esper et al 2002 and Osborn and Briffa 2006, which has been fairly productive, but still not quite complete. I haven’t posted up or discussed all my correspondence by any means. It’s frustrating to look back and see exactly how much time has been consumed in quasi-litigation over data.
The NAS panel relied heavily on articles using unarchived data and I plan to send a letter to Ralph Cicerone, President of NAS, asking him to directly request unarchived data from the various authors. Here is a draft. If anyone can think of things that I’ve forgotten to mention or has other suggestions, I’d welcome them. I want to send the email out in a day or two while matters are still fresh. At that point, I’ll substitute any changes in the version posted here.
Dear Dr Cicerone,
I am currently in the process of analyzing the interesting report of the National Academies of Science panel on Surface Temperature Reconstructions. The panel has, to a considerable extent, relied on studies for which data is unarchived and/or methods are insufficiently described to enable replication. In most cases, I have tried unsuccessfully to obtain this information. Now that the NAS has relied on this information, I request that you request that the information listed below be archived or, failing that, provided to you so that you can forward the information to me.I apologize for involving you in this process. However, I have made diligent and unsuccessful efforts to obtain this information. I believe that a request from you would be appropriate given the recommendations of the NAS panel and might well be effective.
Yours truly, SM
APPENDIX
1. Lonnie Thompson
For all ice cores and pits from Dunde, Guliya, Dasuopu, Puruogangri, Quelccaya, Huascaran and Sajama,
a) isotope and chemistry information by sample;
b) a detailed description of methods used to date ice cores;
For Quelccaya,
c) a list of all organic samples together with radiocarbon dates for those samples that have been dated.
For most sites, Thompson has only archived decadal àŽⳏ18 data for a portion of the core, although much more information should be available. In Thompson’s case, the request to you is made in two capacities: because his results were used by the NAS panel and because his results have recently been published in PNAS.
2. Rosanne D’Arrigo
Rosanne D’Arrigo presented to the NAS panel in March and D’Arrigo et al 2006 was relied upon by the NAS panel. Could you please request the following information:
a) Site chronologies for all sites used in D’Arrigo et al 2006
b) If some of the measurement data exists at WDCP, exact data citations linking the regional groups in D’Arrigo et al 2006 to any archived measurement data. A considerable amount of the data in D’Arrigo et al 2006 appears to be unarchived, and, in such cases, can you request that the original measurement data be archived.
c) Any presently undocumented protocols used to make RCS chronologies from the measurement data.
3. Hegerl et al 2006
Gabrielle Hegerl presented to the NAS panel and Hegerl et al 2006 was relied upon by the panel. Can you request the following information:
a) Identification of the sites used in this study.
b) If the data versions used in Hegerl et al 2006 are currently archived, exact data citations complying with AGU data citation policies i.e. a data citation to a digital file not to a generic print publication.
c) If unarchived data versions are used, the digital versions as used, together with exact provenance.
d) source code, particularly including the step in which confidence intervals in Hegerl et al 2006 (Nature) are calculated.
4. Esper et al 2002
The NAS panel relied on Esper et al 2002. I have had prolonged correspondence with Science, resulting in their recently obtaining most, but not all, of Esper’s data and they seem to have concluded their efforts without finishing the job. Could you request the following from Esper:
a) the Tarvagatny Pass chronology version and measurement data version as used in Esper et al 2002;
b) Confirmation by Esper that the measurement data archived in May 2006 by Lisa Graumlich is the same data as that used in Esper et al 2002, or preferably, the measurement data for the two foxtail sites as used in Esper et al 2002
c) the measurement data as used by Esper for the two foxtail sites.
d) criteria used by Esper to distinguish between linear and nonlinear sites;
e) criteria used by Esper to decide on which cores to remove from a site data set.
f) source code or other detailed methodological information sufficient to produce the foxtail chronologies as used by Esper from the measurement data.
5. Osborn and Briffa
The NAS panel cited Osborn and Briffa 2006 and even illustrated the series used by Osborn and Briffa. In this case, Science has been relatively cooperative, but have been unsuccessful in obtaining full co-operation. Could you obtain the following information used in Briffa 2000 or Osborn and Briffa 2006:
a) measurement data for Tornetrask, Taimyr, Yamal and Jasper, used to source the chronologies used in Osborn and Briffa 2006;
b) reasons why Yamal data was used instead of updated Polar Urals data in Briffa (2000).
6. Michael Mann
In addition to Mann et al 1998, 1999, other articles by Mann (Rutherford et al 2005 and Mann and Jones 2003) were cited by the NAS panel. In respect to Rutherford et al 2005, could you please request the following:
a) Identification of the 387 MXD series used in this study together with WDCP identifications of archived series, where available, and digital versions of any unarchived versions.
In connection with Mann and Jones 2003:
b) What method was used to determine weights for each of the records? What are the weights for each record?
c) Digital versions of any unarchived series.
In connection with Mann et al 1998-1999,
d) Statistical reference for calculation of confidence intervals in MBH99, together with source code for this step;
e) Source code to calculate the number of principal components to retain in tree ring networks, sufficient to yield retentions observed in Vaganov AD1600 and Stahle /SWM AD1700 networks.
f) Source code sufficient to demonstrate the retention of tree ring sites according to the criteria set out in MBH Corrigendum of 2004.
7. Phil Jones
The NAS panel used CRU temperature data as a reference point. Both von Storch and I quoted Phil Jones’ notorious refusal to archive
a) supporting data, including station data;
b) detailed methodological information and/or source codes.
Climate Audit 
37 Comments
Did they actually get and use the data or did they just do a lit review of analysis that relied on the data (which authors refuse to disclose)?
Proposed add to data request (if not already included):
Lower Troposphere vs. Surface Temperatures
Hi Steve,
From my post http://www.climateaudit.org/?p=756, #170:
Steve, I am fairly confident that this paper is just plain wrong – while the linear regression lines of both datasets are similar, the timing of individual temperature changes within the datasets is quite dissimilar.
It would be worthwhile to determine exactly which datasets ae being used and to re-run this analysis.
And thank you for all your great work.
Best regards, Allan
Except for above ambiguity, letter looks pretty good.
1. Would also change your appendix so that it is parallel in structure/wording where-ever possilbe. For instance, in some of the numbered items, you have a specific request. Others you just list items (I assume you want those also).
2. You seem to have a structure in the paras of:
a. Where the NAS used (or relied on analyses from) the source
b. Explanatory comments about attempts to extract the information
c. “Please request…”
I think you can make this more strictly parralel (for instance para 1 doesn’t follow the pattern).
3. I would also add a more specific reference (footnote that cites a page in the report or direct quote) to their policy recommendation on data sharing.
4. Osborne/Briffa and Mann/Rutherford paras seem a bit jumbled. Who specifically should NAS request data from? You can’t expect someone to give you data for a paper that they were not an author of, no?
5. Within the numbered items: make them all nouns/items, not questions (parallelism).
6. Consider dividing into data and methodology. Consider appropriateness of some of the methodology items. Are they really in the nature of sharing data or are they more the sort of thing that should be taken care of via criticism of the papers. (At least consider how it will look to someone other then yourself.)
——-
All of the above is wordsmithing and the letter is good enough to go off. But you asked for it, so you get it.
Steve, point 7 on Jones should include an active request.
TCO, it’s even below wordsmithing, but you might want to remind yourself that it’s the middle L in parallel which is doubled. Just think of parallel bars and it should be easy to remember. Obviously you’re not the only person to have problems with that word,though. I notice that “Parralel” is one of the spellings that Microsoft Word automatically corrects.
Steve,
Good letter, and go for it!
It seems to me that it would be useful to invoke “Good Scientific Practice” if someone can come up with suitable quotes showing that it is a) standard, and b) expected that scientists will produce their data and methods so that others can replicate their results.
Secondly, it would be worth quoting back to Ciccerone the NAS Panel comments relating to these matters from the NAS report.
I’ll second the suggestion of #3 that the structure be more regular. How about a matrix of author/owner, citation, location, data type (eg, tree-ring), series name, object type (eg, data, code), a brief descriptor, original request date, status, etc.?
Add metadata for the series you’ve received in the past, and you’ve nearly got a searchable database.
Seplinlg deos not mtaetr if the fsirt and lsat ltetres are ceorcrt and all the ltetres are terhe.
lol, jae, you are a star!
Its looks like I’m not the only one who likes to inject a bit of humour into this blog from time to time. After all, it’s hard work following a lot of the details on thi sblog and every now and then I’m sure we all appreciate a slight distraction from the serious stuff.
KevinUK
re: #8 Perhaps so, but note that “Parrelel” doesn’t fit your criteria.
Go after the NAS on the glacier “studies” too. The ones cited are suspect.
On another note, the Carbon Credit scamsters are getting cocky here in America. The Democrats took the money once the ideological cover story was sufficiently publicized via Al Gore and global warming stories.
The USA Carbon Credit lobbies are having a “Forum”. My guess it will be much like an reunion for Enron traders.
http://www.srinstitute.com/conf_page.cfm?instance_id=25&web_id=863&pid=470
“..According to a recent report by the International Emissions Trading Association and the World Bank, “CO2 trading was worth more than 10 billion dollars in 2005, 10 times the value of 2004″ and, as reported in a recent American Lawyer article, the market could reach as much as 50 billion dollars.
In the U.S. and beyond, it is no longer a matter of if, but when and how. Understanding the opportunities and the pitfalls could be the difference in grabbing this financially beneficial and socially conscious brass ring or missing the boat. …”
Blair is in California representing British Petroleum–sorry I should have said Britain–conspiring to partner with California to create a carbon credit scheme. Amazing. I guess there is no press inquiry or policy doubts from pundits because Dems and Repubs, every one is in the scam, or can be coopted. Oil man Bush is under little pressure from the Dems, best he does bleat once in a while about the “hydrogen economy” which apparently is helped by new and additional tax cuts to the oil industry he approved.
I also agree with TCO’s comments in #3.
#10 Yes perlalal works. Has to have all the correct letters. You have two r’s and not enough l’s.
…Now that the NAS has relied on this information, it is of paramount importance that these studies are closely examined to determine if their conclusions are robust, or have limitations such as the NAS panel described for Mann’s work. Clearly, it is not currently possible to critically examine studies where the data and methodology are not set out, and the requirement for a clear and public description of data and methods was one of the key findings of the NAS report. I therefore request that you request …
#14 is very good.
One other suggestion: if you can list in a separate appendix all of the requests you’ve made to get the information–i.e., by date and person–it might better make the point that you’ve put in a serious effort to get the information yourself. (That list may come in handy in other contexts too.)
A minor point regarding style: “I therefore request that you request…” would be better as “I therefore ask that you request…”.
More Strunk and Whitian is “Please request”, a savings of four words.
Re #2, the MSU and balloon data show no increase from 1979 to 1995, and then increase. The three surface records (GISS, HadCRUT3, and GHCN) show no increase to 1985, and then increase. So you are correct, the linear trends are similar, but the details are quite different. In particular, the records diverge widely during the ’85-’95 decade.
w.
For the record, CC your email to North.
On further thought, it might be a good idea to CC it also to each of the delinquent archivers, and perhaps to the journals which have published their papers without requiring them to archive their data and/or methodology, and perhaps also tothe US NSA and DOE, and, for what it’s worth, to Boehlert and Barton.
Lee and Peter Hearden, I’m curious about your thoughts on Steve’s letter. Should scientists publicly provide the data, code, methodology and other relevant information to support their papers?
Thanks
Ancillary thoughts:
1. sometimes people publish “public letters” to someone, about something. Perhaps you might call your note a public letter to NAS about delinquent archiving.
2. you might CC it to any journal of which you are aware which publishes paleoclimate papers. This would be a followup activity so as not to delay the initial transmissio of the letter.
Further ancillary thoughts: you might subsequently send it to those to whom the delinquent archives report, i.e. their bosses, and/or their institutions/universities/governmental agency managements, etc, as well as to the editors of any publications which pontificate on “climate change science”.
An additional ancillary thought: you might send copies to other “national academies, such as the UK Royal Society.
David, I don’t believe I’ve ever indicated otherwise – although I do think that in many or most cses, making the data available for request is adequate if there are not relevant public archives to handle the data archiving, storage, maintainance, and verification work. This is becoming less of an issue as more such archives ae created over time.
I also think there can be occasional justifiable cases where a scientist who has developed a difficult to acquire data set could justify delaying release of the data, for a relatively short, reasonable, defined period of time, while finishing and publishing additional initial analyses.
Or if he wants to keep Scooby and those meddling kids from finding out the real secret of the Haunted House.
Re #25 Thanks, Lee. Quite reasonable.
Re #22, don’t see why not – but why the suspicion of these people? I’d try not to distrust people first. I trust Steve when he say is he does this blog on his own. I trust Steve when he say’s he’s not being paid to do it. I trust other people here who say like wise. Why can’t we just trust these scientists? We do people here think them inherently dishonest?
Would you react well to someone (the tax (IRS?) man say) asking you for every detail of your past financial life for decades back and if you can’t find it all calling you a fraud and a liar? I don’t think it’s reasonable to expect scientist to have to hand every notebook, every scrap of paper, everything they’ve ever written – and for critics to draw unfavourable conclusions if that’s not the case.
I’m a terrible horder of stuff myself, but my ‘office’ is consequently piled high with stuff I ought to chuck and things I can’t find for all the other stuff. Don’t expecet all people to hord everything.
#29
Umm… because they stridently refuse to disclose their data and code?
Re #29
I guess I struggle to see the unprovided data and code as being the kinds of things that get misplaced. If I proposed a new interpretation of data, using special statistical techniques and it got incorporated into a UN document, I would keep the data, code and other methodology. If I later wrote supporting papers, I’d keep my data, code and methodology.
Documentation, and providing it when asked, is a regular practice in the workaday world. It’s a healthy habit.
I believe that all sides of this great debate support open access to data, code and methodology, which is great. I asked the question of the most reasoned and articulate “CA skeptics” (Lee and Peter) to confirm my belief and get it in public. The request for open acess is an issue we can all rally around, all sides of the debate, in a joint pursuit of truth.
Thanks!
Peter H, I certainly never assumed that any scientist was “dishonest”. Business statements are audited routinely and auditors do not expect to find problems and 99% of the time, there aren’t any. Auditors do look at scraps of paper. I’ve personally provided original copies to accoutnants. That’s why accountants have orderly systems for filing relevant documents.
I’ve made sure that I archive my own code for publications – and I’m really glad that I did. I keep working on things and change how I do some calculations and it’s easy to lose track of what you did.
The systems that I’ve recommended are already required in economics journals. It’s a “best available practice” and there is no valid reason for paleoclimate scientists to oppose.
#29 PH
The very word PUBLISH means: to provide to the public.
Most scientists would lose their funding and or job if they couldn’t provide evidence of their work. Are you kidding?
The spin used to support these scientists is just mind boggling.
Certainly this HAS TO BE resolved if their papers are guiding the whole freaking world.
Why is anybody so protective of scientists who find Hockey Sticks in every study they conduct when the Hockey Stick is flawed as flawed can be? Why does anyone want to trust them so much? Why does it make anyone feel good if they see anything that supports the belief in AGW? Why does anyone have to make stuff up? This is a fantasy PH:
“”I don’t think it’s reasonable to expect scientist to have to hand every notebook, every scrap of paper, everything they’ve ever written – and for critics to draw unfavourable conclusions if that’s not the case.
It’s mind boggling that even a Congressional Hearing doesn’t matter to some people. RC continues to make fun of it. The arrogance and self serving behavior on top of data manipulation is also a clue to character and honesty if you ask me.
Peter,
My experience of the taxman is that they never call you a liar but just strike out your expenses if you can’t prove them. Why shouldn’t we do that with science papers? Archive your source data and all you calculations and computer code or your paper is struck out.
However if you do lie to the taxman and submit fraudulent data and he finds out you are likely to pay a hefty penalty to avoid a criminal record. Why not have the same system. Climate Change science is a lot more important than most people’s tax return.
The scientific process REQUIRES that all theories and their associated claims be tested to see if they can be falsified. This does not mean that the people who are performing these tests are accusing the original author(s) of being dishonest. This is just part of the process. They just recently spent millions of dollars to test another one of the predictions of the theory of relativity: frame dragging. This does not mean that they thought Einstein was being dishonest, or even that they thought the theory was wrong; it just needed to be tested. And no matter how many times it’s been confirmed, if even just once it is disproven then the theory, or at least that part of it, falls. This is true of any theory, no matter how long it’s been around. That’s science.
Part curiosity, part nudge; did the request get sent?
Thanks for the nudge. I sent this in today with some editing – thanks for the input – cc to North.