Currygate, again! Judith Curry just can't shut up!
At RealClimate, July 29, 2010, comment #467 by Judith Curry with inline replies by Gavin Schmidt:
At the request of the many emails I’m getting, here is one more salvo about trying to remind people of science should be done and how arguments should be conducted, and how disagreements can be resolved, and conflicts avoided.
[Response: For a start, we could stop thinking of a discussion as a series of 'salvos'... - gavin]
Charles Sanders Peirce (from the Wikipedia) outlines four methods of settling opinion and overcoming disagreements, ordered from least to most successful:
1. The method of tenacity (sticking with one’s initial belief) and trying to ignore contrary information.
2. The method of authority, which overcomes disagreements but sometimes brutally.
3. The method of congruity or “what is agreeable to reason,” which depends on taste and fashion in paradigms.
4. The scientific method whereby inquiry regards itself as fallible and continually tests, criticizes, corrects, and improves itself.
Much of the disagreement is often about ambiguity of statements, and these are easily resolved if the situation has not elevated into animosity and conflict. In the course of rapid exchange blogospheric discourse, people tend not to present formal arguments with carefully crafted premises and conclusions drawn using a specified logic (including myself and the scientists that host this blog), its more in discussion mode. I would personally be interested in a blog that consisted only of formal arguments, queries about ambiguities, and formal rebuttals. But that is not what we have here. Further, when trying to develop a specific thesis in a blog comment stream, other comments hone in on ambiguities in one of the premises as an attempt to dismiss the entire thesis. Yes, lets try to eliminate the ambiguities, but more importantly lets try to understand the main thesis in someone’s points. In the blogosphere, when all this is laced with heavy snark and appeal to motive attacks, it is very difficult get anywhere.
[Response: Indeed, that's why we have the peer-reviewed literature. - gavin]
A critical element in avoiding conflict, justifying a thesis, and understanding someone’s statement or thesis is to ask the question “What would have to be the case such that this statement/thesis were true?” And then both the proponent and examiner should ask the reverse question: “What would have to be the case such that this statement/thesis were false?” The general idea is that the fewer positions supporting the idea that the statement/thesis is false, the higher its degree of justification. Verbal ambiguities can easily be resolved in this way. And it’s a good way to clarify scientific debates also. This is called looking at both sides of the argument and actually trying to understand them. Kudos to those of you who wandered over to climateaudit to try to see what was going on over there and understand their arguments.
When there is a great deal of uncertainty and ignorance on a scientific topic (paleo reconstructions certainly qualifies here), the problem arises when we have conflicting “certainties” by two sides. Conflicting certainties arises from differences in chosen assumptions and the natural tendency to be overconfident about how well we know things. Most of this conflict can resolved by acknowledging and understanding the uncertainties.
Conflicts about methodology can be more easily resolved than conflicts about scientific hypotheses (e.g. 1998 is the warmest year in the last millennia), although methodological issues are a key element required to support a scientific hypothesis.
Uncertainty is complex beast, with multiple locations, different natures (imperfections of knowledge vs inherent variability), and different levels ranging from the unrealizable ideal of complete deterministic uncertainty to total ignorance. The IPCC has not done a very thorough job in characterizing uncertainties. In the first IPCC assessment reports, the executive summaries included lists of “we are certain of the following” and “we have confidence that”, and they included a list of four broad areas of uncertainty. For the IPCC third and fourth assessments, Moss and Schneider’s (2000) guidelines were followed, with a common vocabulary to express quantitative levels of confidence based on the amount of evidence (number of sources of information) and the degree of agreement (consensus) among experts. The actual implementation of this guidance in the AR3 and AR4 WG1 Reports adopted a subjective perspective or “judgmental estimates of confidence,” whereby a single term (e.g. “very likely”) characterizes the overall confidence. Now there have been all sorts of critiques of this method in the published literature, but lets accept the method for the sake of argument.
With this in mind, lets examine the following aspects of my statement in #167:
“3. The NAS North et al. report found that the MBH conclusions and “likely” and “very likely” conclusions in the IPCC TAR report were unsupported at that those confidence levels. How the hockey team interpreted the North NAS report as vindicating MBH, seems strange indeed.
This statement is ambiguous, it can be interpreted in several ways. The verbal ways that the strength of the arguments in MBH, the IPCC, and the North report were described differently, I attempted a generalization using words that the readers would identify as confidence levels. The ambiguity could have been resolved with a longer statement that was more clearly worded. Remember, the point of my summary was to describe the overall content and arguments in Montford’s book to support my earlier statement that Tamino had missed much of what the book is about. This statement was just one statement in a post that included many points to support my thesis regarding missing elements in Tamino’s review, my intent was not to reproduce these arguments in any detail or immerse myself in the technical battle on this issue. This ambiguity in my statement is easily clarified, and does not detract from my overall thesis, and does not in any way reflect on the accuracy of Montford’s book.
[Response: But your statement was based on false premises -- that IPCC TAR had made very strong likelihood statements about paleo-reconstructions, that North et al had found them unsupported, etc. Arguments that follow from false premises are just pointless -- they serve no purpose in resolving any point of scientific dispute, and indeed are only generally found when people are making points purely for rhetorical effect. Lesson to be learned? Don't predicate arguments on false premises, and don't be surprised when people correctly distinguish rhetorical tricks from actual discussion. - gavin]
Now on to the real point. My statement below is correct and unambiguous.
4. A direct consequence of the North NAS report is that the conclusions in the IPCC AR4 essentially retracted much of what was in the IPCC TAR regarding the paleo reconstructions. This is the only instance that I know of where the IPCC has reduced a confidence level or simply left out a conclusion that was in a previous IPCC report. This is discussed in the CRU emails.
A reminder, it is this statement in the TAR that is omitted from the AR4:
“It is also likely that, in the Northern Hemisphere, the 1990s was the warmest decade and 1998 the warmest year.” The word “likely” means a confidence level of 66-89%. Gavin and I both agree that this statement is unjustified. We disagree on the significance of this high level of justification in TAR and its retraction in the AR4. This is the only instance of a retraction in confidence from the IPCC. The statements that Gavin cites from Lindzen regarding uncertainties in clouds and aerosols are correct; the IPCC continues to acknowledge a high level of uncertainty and low confidence in these areas, and new uncertainties pop up as the frontier of the science is extended and the description of the nature of the uncertainties becomes more precise. The IPCC has never presented a statement of confidence at the very likely or likely level that has the words “cloud” or “aerosol” in it. The cloud-aerosol forcing/feedback includes much at the border of ignorance, which is acknowledged by the IPCC. But I argue that the ignorance surrounding global/hemispheric global temperature over the past millennia from paleo proxies is a topic where the ignorance level is even greater than the cloud-aerosol issue. I also suspect that the there will be further retraction of the confidence levels in the AR5 regarding global/hemispheric temperature reconstructions. This overconfidence in the IPCC reports on this topic is at the heart of the conflict described in Montford’s book.
[Response: I strongly disagree. Your first statement was not at all specific and therefore ambiguous. AR4 did not 'essentially retract' much of what TAR had to say. What is 'essential'? What is 'much'? This will read by many people with many different opinions and they will infer many different things. My response to you on the other hand was very specific. I did not say that the sentence you quote was 'unjustified', I said that I would have been happier with a 'more likely than not' designation. Saying it was 'unjustified' could equally imply that it was completely wrong or that there was no evidence for it in the slightest -- neither of which I agree with. Again, you are using language in a very ambiguous way which only adds to confusion. I would also add that using your own forecasts of what will be in AR5 as an argument to bolster your case is not useful. Finally, if you think that a difference in 4 words in the IPCC TAR is the only or even an important reason for this mess, you really have not been paying attention. - gavin]
So going back to Charles Sanders Peirce and how to overcome disagreement. On the subject of Montford’s book, Tamino’s review, and the larger issue of the state of the science of paleo reconstructions, what have we seen over the past few days in the blogosphere? CP relies almost exclusively on strategies #1 and #2, my statements rankle so much with Romm because I am an “authority” that he previously referred to. At RC we have seen a mixture of all 4 strategies, with a heavy dose of appeal to motive and ad hom attacks. Given that the RC moderators reject many comments, it is not to their credit that they have been letting these through.
BH tends to #3, they are very polite by blogospheric standards and Montford is amazingly snark-free, but not heavy on scientific arguments on the blog. CA scores highest on #4 (there are elements of the others, but they don’t dominate), they stick to mostly to arguments, evidence, identification and clarification of ambiguities, and ad homs and appeal to motives are snipped. With regards to snark, it is more evident in the main posts at CA than at RC, although the inline comments from McIntyre are relatively snark free, whereas those from Gavin are not. Snark is endemic to the blogosphere, makes it entertaining I guess, to some anyways. But snark neither adds nor detracts from scientific arguments, it merely distracts. So readers interested in the arguments should filter it out, and not keep tallies based on snarky gotchas that are minimally relevant to the argument.
[Response: We snipped a lot of comments that were inappropriate directed at you, this is not perfect, but there is a limit to how much time people have to moderate these things. As for whether people can get past the 'snark', we are going to have to disagree on that. I find continual insinuations of misconduct, lying, and 'hiding the pea under the thimble' offensive. I also find continual misrepresentation tiresome. For instance, I made a comment mentioning that the NAS report had cited Fritts (1976) as a source on the use of the RE statistic (which is true). This was completely misrepresented by SM. I also made a statement about what happens in the no-dendro/no-Tijlander case with the Mann et al. (2008) data and method -- which is true, and yet was interpreted as yet more perfidy from scientists. SM knows the PCA issue is moot, and yet keeps on bringing it up. You may think whatever you like about CA, but as a forum in which truth-seeking can be done, it fails miserably. - gavin]
And finally, since I am a glass half full kind of person, I am trying to see what we might have gained from this exchange across the four blogs. The insults that have been heaped upon me are irrelevant to me, if i want friends i can hang out on facebook. They are irrelevant to the arguments themselves, and are only of relevance if you are using Peirce’s #2 strategy, which isn’t very effective in any event. I am prepared to declare victory if anyone is seriously looking at both sides of the arguments, there are any new readers for Montford’s book, if people have wandered over to climateaudit to check it out, if people (especially the RC principals) are starting to get it that the watchdog auditors (e.g. McIntyre) are different from the merchants of doubt (I’m sure that CP won’t cede this). And most importantly I hope that the dialogue can change regarding uncertainty, to acknowledge that there is a high level of uncertainty in level 3 science (as per my previous post on Funtowicz and Ravetz classification), and still a significant level of uncertainty in level 4 science, and not too much of climate science is actually at level 5. The rebels who dispute the level 4 consensus are not irrational, and you need to differentiate rebels from cranks. An interesting case of this is Roy Spencer (a rebel) currently taking on the cranks that are denying that the greenhouse effect exists.
It would be much easier if the public could just trust the experts to be right. This doesn’t work, and again Feynmann said it best: “Science is the belief in the ignorance of experts. When someone says, ‘Science teaches such and such,’ he is using the word incorrectly. Science doesn’t teach anything; experience teaches it. If they say to you, ‘Science has shown such and such,’ you should ask, ‘How does science show it? How did the scientists find out? How? What? Where?’ It should not be ‘science has shown.’ And you have as much right as anyone else, upon hearing about the experiments (but be patient and listen to all the evidence) to judge whether a sensible conclusion has been arrived at.”
[Response: No-one disagrees with Feynman, so I wish people would stop quoting him as if we have never read this. But this issue has nothing to do with anyone here saying that no-one should ask questions, or that everything that there is to be known is known, or that the IPCC or 'science' is perfect. At workshops, meetings and emails all of the issues are regularly being hashed out and looked at. It should go without saying (but obviously doesn't) that people should ask as many questions as they want, that there is a lot more to be learned, and that neither I, the RC contributors, the IPCC, 'science' or the NAS are perfect. (Though that does not imply that we know nothing.) Again (and this too should go without saying, but apparently it doesn't), no-one in the scientific community is against openness, or transparency or data availability. None of these things are points of contention in the slightest (though neither are they always simple).
What is being pushed back against is the continual barrage of innuendo, accusations of corruption and fraud, and insinuations of misconduct because people had the temerity to do their jobs and publish results which some people do not like. This has happened to Ben Santer, Phil Jones, Mike Mann, and many others and follows in a long line of similar tactics employed by the 'merchants of doubt'. McIntyre might not fall exactly into that mold (almost certainly very different motivations), but he feeds that machine quite willingly -- and not just in relation to MBH: read his posts on Hansen or his accusations against Briffa on the Yamal issue, for instance. Every time there is a puzzle or ambiguity or something he doesn't understand, the first recourse is to accuse the scientists of bad faith. Montford's book is just more of the same paranoia ('the corruption of science'? Really?), and your championing of it simply further poisons the atmosphere of debate. Why is it so hard to realise that you can't have a dialogue between scientists and people who accuse them of lying all the time?
If McIntyre wants to be taken seriously by scientists (and it is not clear that he does), he needs to eschew that kind of nonsense despite the adoring Greek chorus who egg him on for their own reasons. Indeed, in the past you have made exactly the same point. It isn't that these insinuations get in the way of understanding his points, it is that they appear to be his points. As I said earlier, science has indeed set up structures (peer-review, etc.) that allow arguments to be resolved efficiently. Those structures work remarkably well. It is time the 'auditors' started to use them. - gavin]
Let’s look at the issue of belief in three points straight from the title of Montford’s book:
* is the “Hockey stick” an illusion?
* is “climategate” significant?
* is [any part of] science corrupt?
On the method of “tenacity”, Peirce explained his meaning:
Once you have formed any opinion on a subject, it’s a perfectly natural human tendency to cling to that opinion, and “both sides” do it here. From the start most of us with some scientific background and trust in the methods of peer review would have tended to the “No” answer on each of the three questions, though many of us may have had doubts on each point. Clearly on the McIntyre/Montford side, the tendency was to answer “Yes”.
The question regarding Peirce’s classification is to what degree those beliefs were *fixed* by this method of simply clinging to them, despite contrary evidence. Tamino’s review here and past discussions (Mann 2008 in particular, but even all the other reconstructions like Loehle’s) clearly show a certain “robustness” in the “hockey stick” form and the conclusion regarding recent warmth being highly anomalous. What is the evidence on the other side? There were clearly some mistakes made in the analyses (in Mann’s papers, and far worse in Loehle’s) but there doesn’t seem to be any modern reconstruction published in the peer reviewed literature that shows anything other than essentially the picture in MBH98.
So the weight of the evidence on question 1 is very heavily in favor of those who fixed belief in the “No” side of the question, and heavily against those on the “Yes”. Have any of those on the “Yes” side changed their minds in regard to this evidence? If not, then they are clearly stuck in Peirce’s “tenacity” mindset.
Similarly on climategate and corruption, we’ve now had 5 reviews that found at worst some poor communication and data sharing practices. The weight of the evidence is very strongly on the “No” side of these questions. Have any of the “Yes” folks changed their minds as a result? There are some examples: The Guardian’s George Monbiot, for instance, at first thought climategate was significant and called for Phil Jones to resign, but has now stepped back from that (his concern seemed to be chiefly from a deep faith in the importance of FOI-type laws).
Peirce’s method of authority referred to imposition of belief by government or other institutional agents. On the above three questions and in the present discussion it doesn’t seem to me to apply at all. I don’t believe he was referring to “authority” in the sense of expertise; in some sense the role of the IPCC in fixing belief around climate science is similar to Peirce’s “authority”, but it has no enforcement power and to me it seems far more like a step in the process of fixation and communication of scientific information, part of the publishing process, than anything like what Peirce was talking about in method 2.
Peirce’s last two methods of fixation are perhaps the most interesting for scientific discussion: that which is “agreeable to reason” vs that which “coincides with fact”. What is the source of “fact”? Observation of the world around us – raw data, collected and prepared according to scientific standards of care, self-doubt, and open honesty. On top of that data we have analysis and interpretation, which again require scientific care.
As an example of the “a priori” “agreeable to reason” method, Peirce takes an interesting example – classical economics:
In this context, there are several groups that are practicing the scientific method in the sense of actually going out into the world and making observations. The people who study tree rings, those who study borehole temperatures, those who set up and monitor temperature stations, those who study glaciers, sometimes risking their lives, are gathering that basic factual data about the world. On top of that are folks like the CRU and Mike Mann, collecting that data together and analyzing it to see what it can tell us, what the larger-scale picture might be.
That scientific method can have bearing on only the first question: is the “hockey stick” an illusion, an artefact, an accident of the way the work was done? And the only way to answer differently is to do your own reconstruction, go back to the raw data and see what difference it makes. As Tamino illustrates here, that seems to have been done repeatedly by others and they come back with the same answer. So the scientific method, as far as it seems to have been applied at all here, clearly shows a “No” answer to the first question.
But the second and third questions are not matters where we can apply any observations of the non-human natural world to help fix our opinions. These are questions about what people said, what they meant, what in their hearts they honestly were feeling. Since the internal workings of another human being are fundamentally subjective and cannot be determined by any objective method, our only resort on this question, our best option, is to appeals to reason. The “facts” on scientific corruption may be eternally in dispute, but what conforms best to our understandings of the people involved, of reason in this matter? Conspiracy theories can certainly be logically self-consistent, but they are fundamentally destructive to human relationships and progress. The fundamental foundation of all of that has to return to the one question that can be answered by science here: “is the hockey stick an illusion”?
If you want a scientific discussion, that is the only question of relevance. And Tamino addressed precisely that question in his review here. The answer is, No.
Comment #512 by Arthur Smith — 30 July 2010 @ 10:12 AM http://www.realclimate.org/index.php/archives/2010/07/the-montford-delusion/