Gavin Schmidt of Real Climate: The Guardian disappoints

The Guardian disappoints

Real Climate — gavin @ 23 February 2010

Over the last few weeks or so the UK Guardian (who occasionally reprint our posts) has published a 12-part series about the stolen CRU emails by Fred Pearce that are well below the normal Guardian standards of reporting. We delineate some of the errors and misrepresentations below. While this has to be seen on a backdrop of an almost complete collapse in reporting standards across the UK media on the issue of climate change, it can’t be excused on the basis that the Mail or the Times is just as bad. As a long-time Guardian reader and avid Guardian crossword puzzle solver, I’m extremely unhappy writing this post, but the pathologies of media reporting on this issue have become too big to ignore.

We highlight issues with three of the articles below, which revisit a number of zombie arguments that have been doing the rounds of the sceptic blogs for years. Two follow-up pieces will deal with two further parts of the series. Hopefully some of the more egregious factual errors can be fixed as part of a ‘group experiment‘ in improving the stories, though the larger misconceptions probably can’t be (and readers should feel free to use this information to comment on the articles directly). Why the Guardian is asking for group input after the stories were published instead of before is however a puzzle. Some of the other pieces in this series are fine, which makes the ones that get it so wrong all the more puzzling. The errors consist of mistakes in the basic science, misunderstandings of scientific practice, more out of context quotes and some specific issues that are relatively new. (In the text below, quotes from the articles are in italics).

Part 3: Hockey Sticks

Some of the more egregious confusions and errors were in the third part of the series. In this part, a number of issues that were being discussed among the paleo-community in 1999 were horribly mixed up. For instance, there was a claim that arguments on the zeroth-order draft of the 2001 IPCC report were based on Briffa’s reconstruction showed the 11th century as being almost as warm as the 20th century, while Mann’s graph found little sign of the earlier warming. But this is simply untrue since at the time Briffa’s curve only went back to 1400 AD (not the 11th Century) and the discussions had nothing to do with the medieval warm period, but rather the amount of multi-decadal variability in the three different reconstructions then available. This was corrected in the online edition, but the description of the dispute in the article is still very confused.
That discussion was conflated with a completely separate April 1999 issue based on a disagreement about a perspectives piece in Science (which appeared as Briffa and Osborn, 1999) and which was in any case amicably resolved.

That discussion is then further confused with the discussions about the framing of the SPM text which despite Pearce claiming that ‘the emails reveal how deeply controversial it was at the time, did not get discussed in the emails at all. And while the article claimed that the uncertainty was not discussed in the IPCC report, the discussion in Chapter 2 was actually quite extensive.

Part 5: Chinese weather stations

This piece concerns the response of Phil Jones at CRU to a FOI request for data that had been used in a 1990 paper on the urban heat island (UHI). This now-20 year old paper was an early attempt to try and assess the possible magnitude of the UHI impact on the global temperature records. (Note that this is not the same as thinking that UHI does not exist).

Starting from the headline “Leaked climate change emails scientist ‘hid’ data flaws” on down, the article is full of misrepresentations. To start with, the data in question (and presumably it’s flaws) were not hidden by anyone, but rather had been put on the CRU server in 2007 response to a FOI request. Hardly ‘hidden’. Exactly contrary to the truth of the matter, the article incorrectly asserted that ‘Jones withheld the information requested under freedom of information laws’.

These data assumed a much greater importance later in 2007 when they were used for a completely unsubstantiated claim of ‘fabrication’ and ‘fraud’ against Wei-Chyung Wang (a co-author on the paper) at SUNY Albany by a certain Douglas Keenan. These charges were found by the university to be baseless in 2009 and the matter was dropped. However, the Guardian noted that a couple of the emails mentioned the issue, and that one in particular had Tom Wigley asking Phil Jones about the situation. Curiously enough, Phil Jones’ response was not part of the archive, and Wigley’s current thoughts on the subject (presumably that have been informed by Jones’ answers) were not reported.

Pearce describes this conversation saying that ‘new information brought to light today indicates at least one senior colleague had serious concerns about the affair‘. However, Tom Wigley has subsequently passed on later conversations to me showing very clearly that he did not support Keenan’s allegations of ‘fabrication’ and the implication that he does here are very misleading. Indeed, the statement that ‘Tom Wigley, harboured grave doubts about the cover-up‘ is completely false. There was no ‘cover-up’; the email was written two years after the data had been posted online.

The line in the 1990 paper that has apparently caused the furore is the following:

“The stations were selected on the basis of station history: we chose those with few, if any, changes in instrumentation, location or observation times.”

For fraud to have been proven, it would have been necessary to show that Wang – at the time of the 1990 paper – deliberately misled in the line as it was written. It would not be enough to show that the statement was mistaken because of incomplete histories available to him at that time, nor that some stations had in fact moved. The statement is a declaration of a good faith effort to pick suitable stations. Instead, you would have to demonstrate that Wang was aware of substantial and important moves that made a material difference and deliberately concealed this fact. And for this there is absolutely no evidence. Keenan’s assumption of fabrication is merely that, an assumption.

Wigley’s ‘grave doubts’ were a suggestion that the key line be rewritten as

“Where possible, stations were chosen on the basis of station histories and/or local knowledge: selected stations have relatively few, if any, changes in instrumentation, location, or observation times”

A change that doesn’t undermine the paper in the slightest, and would hardly be likely to set the blogosphere aflame.

Quite frankly this whole allegation is absurd – why would anyone do this? All the authors involved have written many papers on the problems in the temperature record and on Urban Heat Islands in general, and even in China. Indeed the story here is that information was provided under FOI rules, and that it was not used to constructively examine the science, but rather to provide ammunition for baseless accusations that led to pointless university inquiries into alleged misconduct. That might be a good reason for why FOI requests are now being viewed with suspicion.

Other claims that this ‘may yet result in a significant revision of a scientific paper that is still cited by the UN’s top climate science body‘ . and that ‘what data is available suggests that the findings are fundamentally flawed‘ are simply made up. The findings of the 1990 paper was that UHI was unlikely to be contaminating the global temperature records in any significant way has been upheld by any number of additional studies in the 20 years since it was published. Oceans are not warming because of UHI, spring is not coming earlier because of UHI, and indeed, glaciers are not melting because of UHI (they are of course melting, recent news reports notwithstanding). No evidence of significant UHI contamination was found by Parker (2004, 2006), the record from GISTEMP which applies a different UHI correction than HadCRUT does not differ substantially at the global or regional scale. Other studies by Peterson, Jones, and others all show similar results. Even the more recent analyses of the Chinese stations themselves and even in an environment where urbanisation is happening faster than ever, UHI effects are still small (Jones et al. 2008).

As an aside, Keenan has made a cottage industry of accusing people of fraud whenever someone writes a paper of which he disapproves. He has attempted to get the FBI to investigate Mike Mann, pursued a vendetta against a Queen’s University Belfast researcher, and has harassed a French graduate student with fraud accusations based on completely legitimate choices in data handling. More recently Keenan, who contacted Wigley after having seen the email mentioned in the Pearce story, came to realise that Wigley was not in agreement with his unjustified allegations of ‘fraud’. In response, Keenan replied (in an email dated Jan 10, 2010) that:

... this has encouraged me to check a few of your publications: some are so incompetent that they seem to be criminally negligent.

Sincerely, Doug

This kind of knee-jerk presumption of misconduct (and criminal misconduct at that) when people disagree with you has no place in the scientific discourse, and serves only to poison scientific debate. Indeed, Jones adds in one of the emails: “I’d be far happier if they would write some papers and act in the normal way. I’d know how to respond to that”. For the Guardian to dignify this kind of behaviour – especially after the charges had been investigated and dismissed – is unconscionable and a public apology should be forthcoming to Jones, Wigley and Wang.

Part 6: Peer review

The discussion of peer review is the most replete with basic misconceptions about the scientific process. Pearce appears to conflate any rejection of a paper or even a negative review for any reason as a prima facie case of mainstream climate scientists … censoring their critics. But in none of the cases highlighted were anyone’s view ‘censored’. To have your opinion published in peer-reviewed literature is not some fundamental right – it is a privilege that depends on your ability to do the analysis and the marshal the logical arguments and data to support your point.

Pearce, surprisingly for someone who has been on a science beat for a long time, states that peer review is the supposed gold standard of scientific merit. This is not the case at all. As we’ve outlined in many articles, peer review is just a first (necessary) step towards scientific acceptance and as the number of badly flawed papers that do appear in the literature attest, it is no guarantee of merit. For it to work of course there need to be some standards that should ideally be met, and this will lead to the rejection of some submissions. Thus automatically equating rejections of bad submissions with squashing of ‘dissent’ is like assuming that anyone who gets an F on a test is being unfairly discriminated against.

Pearce also declares that the mere act of reviewing a paper that is critical of your own work is mired in ‘conflicts of interest that would not be allowed in most professions‘. This is wrong on multiple levels. First of all, peer review of the literature is hardly unique to climate science, and so his claim about improper conflicts of interest is an accusation against the whole of science, not just climatology. Secondly, he confuses the role of the reviewer with that of the editor. Editors often solicit reviews of a critical comment directly from those being criticised, since that is often the easiest way to judge whether the critique is substantive. That is not the same as giving the right of veto to the criticised authors since, of course, it’s the editor’s job to weigh the different reviews from different sources, and use their own judgment as to the merits of the critique. Not asking the original authors for comment can certainly be (and has been) problematic and unfair to them. The problems most often arise – such as in Soon and Baliunas (2003) or McIntyre and McKitrick (2003;2005) when the criticised authors are not involved at all.

In the cases mentioned in this article, there is absolutely no evidence of unfair discrimination. Indeed, in one case of a submission by Lars Kamel, the reasons for rejection are obvious and Pearce appears not to know what the criteria for acceptance even are. He states that “the finding sounded important, but his paper was rejected by Geophysical Research Letters (GRL) that year“. But papers are not accepted or rejected because a finding ’sounds important’, but because that finding is backed up by analysis and logic while acknowledging the prior work on the topic. In this case, the author did not “however, justify that conclusion with any data or analysis“, and so a rejected manuscript would have been very likely, regardless of who the reviewers were. Similarly, the assumption that “some would have recommended publication” purely because it called into question previous work is unsupportable as a general rule. Filling the literature with papers ‘just asking questions’ that ’sound important’ but not demonstrating any actual results is a recipe for wasting everyone’s time with poorly thought out, and even mendacious, critiques of mainstream science from HIV-denial to perpetual motion machines. Papers in the technical literature are not just opinion.

Pearce also assumes (without evidence) that Kamel was discriminated against because Jones “would certainly have been aware of Kamel’s [negative] views about mainstream climate research“. But why should this be assumed? Most scientists (luckily) go through their whole career without wasting their time investigating and cataloguing the cranks in their field. Some climate sceptics get addressed here on RC a fair bit, but it would be a big mistake to think that these people, particularly the more obscure ones, are the subject of water cooler conversations at climate research labs across the world. Indeed, I can find no reference to Kamel on RC at all and I was unaware of his peculiar views until this story emerged. Why Jones should be assumed to omniscient on this topic is unclear.

Pearce quotes McIntyre discussing “CRU’s policies of obstructing critical articles in the peer-reviewed literature” slowing the resolution of unspecified “issues”. This is simply disingenuous – what papers have been obstructed that would have resolved what issues? We are unaware of any such papers, and certainly none from McIntyre. Prior therefore to declaring that “evidence, flawed though it might be, is actively being kept out of the journals” it behoves Pearce to actually find such evidence. Otherwise, the simple non-appearance of these mythical critiques is apparently proof of the corruption of the peer review process.

As an additional example of problematic practice, Pearce highlights a June 2003 email from Keith Briffa, who as an editor ‘emailed fellow tree-ring researcher Edward Cook, a researcher at Lamont-Doherty Earth Observatory in New York, saying: “Confidentially I now need a hard and if required extensive case for rejecting [an unnamed paper] to ­support Dave Stahle’s and really as soon as you can.”‘. However, without context this is meaningless. People often sign reviews and this could well have been a second go around on a particular paper whose first round reviews would have been seen by everyone concerned. Briffa (like many editors) can have a feeling that a paper should be rejected for multiple reasons but would like to have the reasons gone into in some detail, mostly for the benefit of the authors. This is one reason why reviewing bad papers is so much more work than good ones. Quoting this as if it absolutely demonstrated bad faith or misconduct is simply a smear.

Pearce then accuses Cook of some unjustified quid-pro-quo because he wanted to use some of Briffa’s data to assess the practical implications of a new analysis technique, that Pearce interprets as “attacking his own tree-ring work“. However, this too is a misreading. The work in question has subsequently been revised and the authors themselves have said that the current submission is improved over the initial submission. It goes along with the overall point made above, that pure criticism is not particularly useful – it is much better to demonstrate that some technical point actually matters. This is what Cook appears to be asking for help to demonstrate.

The article then moves on to the issue of the 2003 Soon and Baliunas paper in Climate Research. Pearce nowhere acknowledges that it is (and was) widely regarded as a complete failure of the peer review system. Six (very independent minded) editors resigned from the journal because of the publisher’s inaction on tightening up peer review standards and even the publisher himself declared that the paper’s conclusions were not supported by the data or analysis of the authors. Is this not germane?

Pearce suggests that the reaction to the demonstrably low standards at Climate Research involved “improper pressure“. This has no validity whatsoever. The suggestion was made that maybe people should not submit work to the journal or cite work that appeared there. But how can a suggestion made among colleagues and not transmitted more widely be ‘pressure’ of any sort? People have their impressions about journals determined by many factors, and if they are seen to be publishing bad papers, that will be noted. Compare the reputations of Science and E&E for instance. Which would you rather be published in if you had a good paper?

The one email that Pearce declares “means what it seems to mean” refers to the declaration (along with exclamation point) that Jones would “redefine peer-review!” rather than include two flawed papers in the AR4 report. But it should be obvious that no one gets to redefine what ‘peer reviewed’ means, and the exclamation point underlines the fact that this was hyperbole. The two papers referred to (McKitrick & Michaels, 2004; Kalnay & Cai, 2003)) were indeed discussed in Chapter 2 of AR4 as the contributing lead author of that chapter Trenberth rightly pointed out. As an aside neither have stood the test the time.

The problem with lapses in peer review (which will inevitably occur) is that they are sometimes systematic, indicating a more institutional problem instead of simply an unfortunate combination of poor reviewers and a busy editor. This appeared to occur at Geophysical Research Letters over the period 2005-2006. There was a string of bad papers published – ones that did not properly support their conclusions and made basic errors in the science. For instance, Douglass and Knox (2005), Douglass, Patel and Knox (2005), Douglass, Pearson and Singer (2004), Douglass, Pearson, Singer, Knappenberger and Michaels (2004), and Loáiciga (2006).

Science is indeed a ’self-correcting’ process, but someone has to do that correcting, and scientists do get frustrated when they have to spend weeks dealing with the aftermath of bad papers in the media and putting together the comments that almost every single one of these papers generated. (For amusement and for an example of the lack of standards being talked about, look at the response of Bjornsson et al. to the Douglass, Patel and Knox paper).

Are scientists supposed not to notice these patterns? Or never discuss them among colleagues? The implication that the mere discussion of the situation is somehow a corruption of the peer review process is completely unjustified. Peer review only holds the status it does because scientists are on guard against failures in the system and try to correct them when they occur.

Update: Coincidentally, David Adams on the Guardian makes many of the same points as we do.
In two follow-up pieces we will host a letter from Ben Santer on Part 7 and on the skewed reporting of the ‘Yamal‘ issue in Part 9.

Link:  http://www.realclimate.org/index.php/archives/2010/02/the-guardian-disappoints/