[JohnA: For reasons that I don’t understand, posting comments became nearly impossible for some people for a short while. This comment from Willis I thought should be captured for future reference so I made it into a post.]
Willis Eschenbach writes:
Well, on the 4th of this month I posted a couple of questions on the Climate of the Past discussion site. After 10 days (on the 15th), I got tired of waiting for an answer, and I posted the following:
I had made the foolish assumption that this discussion board was an opportunity to discuss the paper with the authors. However, they seem to be curiously unwilling to respond to my questions. If they are not willing to answer questions … what is the point of having a discussion forum at all?
w.
In response, today (the 21st) I got a very courteous answer from Hugues Goosse, the Editor in Chief. He wrote:
Dear Willis Eschenbach,
I contact you as the editor in charge of the paper submitted to Climate of the Past Discussion, entitled “Millennial temperature reconstruction intercomparison and evaluation” (Juckes et al.) which is currently in the Interactive Discussion phase. Short Comment (SC) such as the one you have posted on the Interactive Discussion site of Climate of the past (http://www.cosis.net/members/journals/df/article.php?a_id=4661 ) are fully part of the Open Discussion Phase and I clearly understand you wish to get a quick answer to this comment. On the other hand, according to Climate of the Past Discussion rules, a quick answer of the authors is not required. An answer to all the relevant short comments could occur at any time during the Open Discussion Phase (8 weeks) or the Final Response Phase (4 weeks, expandable if required by the authors). I have personally contacted the authors to ask them if it would be possible to get a response sooner than the limit imposed by the rules but this will remain their choice.
If you have additional questions about to Climate of the Past rules, do not hesitate to contact me or one of the chief editors of the journal.
Sincerely
Hugues Goosse
I have answered him as follows:
Dear Hugues Goosse:
Thank you kindly for taking the time to respond to my posting at Climate of the Past. It would be helpful if you could post these rules prominently in the discussion area, so that others will not be misled into thinking that there might be a free and frank interchange on your web site about the ideas put forth in the paper.
In addition, it would be even more helpful if the rules were changed so that a more interactive discussion could take place. One of the benefits of the internet is that ideas can be exchanged and topics discussed in short order. I am quite fearful that by the time the authors get around to answering the questions, that the time frame will be elapsed and the discussion will be closed.
A number of questions have been asked at Climate Audit about this report. Dr. Juckes has graciously replied to some of these questions, but in the main they remain unanswered. As these questions strongly impact the scientific value of the document, and they have not been answered in either forum, the quality of the work is very much in question.
As Editor in Charge, it is your responsibility, not the authors, to see that these questions are answered. Although the authors may choose to wait until the end of the period in order to forestall serious discussion of the paper, it is your responsibility to choose whether or not to print it. I encourage you to go to ClimateAudit, where there are a number of threads regarding this paper, to acquaint yourself with some of the difficulties with their work.
Relevant pages there are:
(Juckes and 99.98% Significance)
(Juckes and the Indigirka River Alter Ego)
(Potential Academic Misconduct by the Euro Team)
(Juckes and the Divergence Problem)
(The Euro Team and the SWM Network)
(Your Comments on Juckes Omnibus)
(Juckes and the MITRIE Project)
(Juckes and Reconstruction #9)
(uckes and the Esper and Jones CVMs)
(Re-Fried Greenland Ice Cores)
(The Euro Hockey Team and Yamal)
As you can see by the number of threads about the paper, a number of different, very troubling problem areas have been identified which bring into question the scientific soundness of the paper. These problems include, among others, misidentification of data, one data series being used twice under different names, using older data when newer data is available without justification, not following stated a priori rules for proxy selection, having a calibration period without a verification period, opportunistic flipping of principal components, and lack of adequate testing for robustness. They also neglect to justify their inclusion of foxtail tree ring width proxies, in total disregard of the explicit recommendation of the NAS Panel. In a paper which claims to examine the field of historical temperature proxies in general, this is a staggering omission.
In general, there have been no answers to these questions. I trust that you will take into account the demonstrated unwillingness of the authors to answer the hard questions about their work into account when you decide whether to print their document or not.
In addition, I would strongly suggest that you require the authors to delete the comments about the availability of code and data from Steve McIntyre, or you might find yourself in a very ugly legal dispute. Several of the co-authors of the paper have made it an unfortunate habit to refuse to provide data and methodology to other scientists. To claim that Steve McIntyre is guilty of this (which is absolutely not true), while ignoring the proven misdeeds of some of the co-authors of the paper in this regard, will open both you and the authors to the possibility of a civil suit for libel. I would be extremely cautious in this regard, were I in your shoes.
Finally, I appreciate your taking the time to inform me of the laxity of your rules regarding the author’s responses to questions, it makes the situation clear. I was looking for interaction, and it appears that I will be lucky to get action … but I will take whatever I can get.
Warmest regards,
w.
Hugues Goosse has replied as follows:
Dear Willis Eschenbach,
The rules of Climate of the Past are clearly displayed at: http://www.copernicus.org/EGU/cp/interactive.html
There is access to these rules from the left menu when in the discussion area, so anyone could consult them easily I guess. In contrast to the majority of scientific journals, for Climate of the Past everyone has access to information on the peer-review process and any member of the scientific community could publish Short Comments, as you did. Compared to the classical review process, the goal is to foster scientific discussion and to enhance the quality control of the paper. However, the Short Comments should be of substantial nature. Answering those comments, as well as the ones of the Anonymous Referees, could require a large amount of work: to perform additional analyses or experiments, to read scientific papers… Consequently, It is not possible to oblige the authors to have a quick answer to all the comments. We could thus not have rapid blog-style conversations (but, as you know, there is several blogs where this is possible). On the other hand, at the end of the process, the authors must have answered adequately all the relevant comments. This is the role of the editors to check that it is actually the case. If some problems are remaining, the editors should require minor or major changes before publication or should consider that the work could not be published because substantial problems have been raised.
I hope that this brief mail has helped you to have a clearer idea of the philosophy of the public discussion of Climate of the Past.
Sincerely
Hugues Goosse
My conclusion from this is that we should collect all of the unanswered questions and inconsistencies in the paper, polish them up, and put them on the Climate of the Past discussion site. That way, they will be part of the official peer review process.
My best to all, and my particular thanks to Hugues Goosse.
w.
Climate Audit 
16 Comments
Willis, you say:
Exactly! And, I would add, be polite with the Editor! A few weeks may seem like a long time to respond, because you are used to CA’s rapid pace, but it’s best that the authors take the time to adequately respond to detailed and precise comments. Like everybody else, they ARE busy doing other things. When you get a paper published, it is typically the result of the work you were doing six months to a year ago, and by the time it appears in print or on line, you are working on other projects, and responding to comments is not necessarily the highest priority. If you want your comments to be taken seriously, you’ve got to play the game and respect the process, because all the scientists who publish do it, and they don’t necessariy like it either!
Have a nice day!
I’d have to mostly agree with Francois on this one. I’d have to say that the immediacy of the Internet is alien to the peer review process, and its difficult to say where the blame really lies. How to tell between a researcher ducking tough questions and one that is simply very busy?
Personally I think the peer review process is lackadaisical and haphazard and needs urgent reform. The anonymous peer review system is deeply flawed when cliques appear as has happened in climate science.
There is no reward for review. It’s all work, no pay. Also, no repercussions for a faulty review. With no carrot & no stick, what do you expect as an outcome? Reform is needed, and this is widely recognized form within. Why doesn’t it happen? Good question.
I am glad you said something Willis. I have been checking to see if they addressed your input yet. Also, early on I read the rules and guidelines and was curious to see how they’d treat SteveM or ClimateAudit in general because the guidelines for submitting are to only “those in the scientific community”. I also wondered if this was used as a loop hole or excuse as to why that paper contained falsehoods about SteveM’s data being available etc, and why the author is so rude to him in general and just didn’t contact SteveM directly.
W.,
I think you have made clear why Dr. Juckes did not answer all questions. Seventeen threads have been opened! It just impossible for him to answer them all and it would be unfair to demand that. Therefore, don’t think it would be a good idea to collect all unanswered questions and dump them on the Climate and the Past discussion site. It would be silly to come up with 50 questions, say. Be realistic, make a selection and come with a reasonable number of questions. Not all of the questions are so good/relevant and asking stupid questions is no contribution to science. It will just irritate the scientist and cost them valuable time.
#5. gb – I recognize that there have been lots of threads. I opened one thread “Juckes Omnibus” to provide Juckes with an opportunity to deal with one set of questions and not require him to check other threads.
If he wants to check other threads, he’s obviously welcome. They’ve been paid over $100,000 for their article so it’s not unreasonable to expect him to spend a little time on the only substantive discussion that this article is ever likely to receive.
I don’t think that the reason Dr. Juckes has not answered the questions is because there are 17 threads. I’m quite able to keep up with the 17 threads, so is bender, so are a host of us, and we’re not getting paid a very large salary to do it.
There’s nine authors on the paper, for goodness sake. Two threads per author. If they were serious about this, you’d think they would find it simple.
But they’re not serious about any of this, as far as I can tell. The reason there’s 17 threads on this question is that they’ve written a paper with a lot of holes in it, and then refused to answer questions about it. The more they refuse to answer, the deeper people have dug, so we have 17 threads.
If they would ANSWER THE QUESTIONS, we could CLOSE THE THREADS …
w.
There seems to be a reply from Martin Jukes. I like the statement
So temperature rises in the last twenty five years must be “After Present”. It may explain why some proxies have not been updated.
#7 Willis,
Come on, there is no obligation whatsoever for Juckes or anyone of his coauthors to reply to comments made on a blog! Don’t overestimate the importance of CA in the scientific community! They can just ignore you completely if they want to. But they would be (professionally) obligated to give at least some sort of reply to a detailed comment posted on Climate of the Past.
I would politely and as scientifically accurate as possible and using scientific terminology, keep restating the problems in the data.
Eventually, someone (the editors probably) will make the call that these concerns need to be addressed.
And if they aren’t, the paper will be dropped. These are serious enough errors that they can’t be ignored.
But your comments Willis have too much negative “emotion” in them. Polite and scientific is the way to get results.
Jeff W., you say in #10:
Of course, you are 100% correct. I get frustrated at the obstinance of these folks, many of whom are on the public payroll, regarding answering questions. Sometimes I feel like I’m talking to the wall. We’ve pointed out a host of problems with the paper, and have gotten evasion, silence, and insults.
But your path is 100% correct when you say:
My thanks for your calm advice.
Francis, in #9 you say:
I agree there is no obligation as you describe. However, I was not referring to them not answering questions made here at CA. I was referring to them not replying to comments at the CoP discussion site, which is their online peer review site. As the Editor of CoP indicated, he expects those questions to be answered.
And while there is no obligation for them to answer questions here, it is certainly in their best interest to do so … as you know from asking some of them, the questions here tend to be serious, substantial, focused, and aimed at very real problems in the work in question. If I had a paper in hand that I wanted to publish, I’d be overjoyed to have the CA auditors take a look at it here before I possibly made a fool of myself in public …
w.
Re: #9. There’s no obligation but it’s in their best interests to bring this debate to a close if they genuinely believe there’s is the correct approach to reconstructing past temperatures. I’m not expert in climate research and as a member of the public I don’t have free access to the papers and journals where important studies are published.
Forgive me for my previous spelling mistake: one downside of blogs is the lack of edit buttons. 😉
Willis,
I find it very odd that Juckes last reply on the discussion site says:
” This is not the place to discuss what has happened in other papers and other journals. ”
Because isn’t that exactly what this paper is about which is titled “Interactive comment on “Millennial temperature reconstruction intercomparison and evaluation” ???
I don’t blame you for submitting your last comment. What a joke!!
FYI, Referee #1 has posted a comment at CoP.
Click to access cpd-2-S689.pdf
bump
Willis, I see you are a real optimist:
How wrong you, me, and several other people were…