Andrew Revkin, Louis A. Derry, Dot Earth, NYT: On Peer Review and (Climate) Progress

On Peer Review and (Climate) Progress

by ANDREW C. REVKIN, DOT EARTH, FEBRUARY 11, 2011

Louis A. Derry, a Cornell researcher and the editor of the journal Geochemistry, Geophysics, Geosystems, provides a valuable look inside the peer review process in a comment on my piece assessing the debate over Antarctic temperature trends. It merits space here as a “Your Dot” contribution (a feature aimed at encouraging constructive non-anonymous discussions).

My favorite line reflects on the importance of placing scientific debates on the edges of understanding in the context of established knowledge: “Disagreement about how to model  the flight of a Frisbee correctly doesn’t imply that basic aerodynamics are wrong.”

Here’s the full comment:

I am an editor for a major Earth science journal, and I would like to make a few comments about the “normal” review and decision process for a paper. Unsurprisingly, there are a lot of comments here and at ClimateAudit and Realclimate that misunderstand some basics. No shame on misunderstanding, but it’s just not right to make accusations of dishonesty on that basis. Very briefly (to keep this post readable):

1. Editors make final decisions. Reviewers make recommendations only.

2. It is common for a submission that critiques previous work to be sent to the author of the critiqued work for review. 2a. That emphatically does NOT mean the reviewer has veto power. It means that his/her opinion is worth having. Such a choice is usually balanced by reviewers that editors believe are reasonably independent, and the review of the critiqued is weighted accordingly. Suggestions that asking Steig to review O’Donnell was somehow unethical are utterly without support in normal scientific practice. Obviously, Steig did not have veto power over O’Donnell’s paper.

3. The fact that O”Donnell’s paper went through several rounds of review is absolutely unsurprising and unexceptional. Many papers on far less public topics do the same.

4. Some have questioned why Stieig 09 got “more” visibility than O’Donnell 10. The answer is simple. Steig had a “result,” O'Donnell had a technical criticism of methodology. As O’Donnell has repeatedly written, he wasn’t trying to address the question of whether Antarctica had warmed, but only argue that Steig’s methodology was wrong. Whether Steig’s result is robust or not, it is often the case that papers with a “result” get more public visibility than technical discussions of statistical significance. If you accept O’Donnell’s work, it still sheds very limited light on the question of whether Antarctica is warming. It really only discusses how well a particular methodology that does try and address that question works.

Finally, Revkin’s point that the Steig vs. O’Donnell debate is not unusual in the progress of science and does not have much of anything to say about the majority of the evidence is correct. Disagreement about how to model the flight of a Frisbee correctly doesn’t imply that basic aerodynamics are wrong. Disagreement about how many EOFs [ empirical orthogonal functions] to use to model Antarctic [temperature] changes doesn’t imply that climate physics is wrong.

 http://dotearth.blogs.nytimes.com/2011/02/11/on-peer-review-and-climate-progress/