Replication and due diligence, Wegman style
Today I continue my examination of the key analysis section of the Wegman report on the Mann et al “hockey stick” temperature reconstruction, which uncritically rehashed Steve McIntyre and Ross McKitrick’s purported demonstration of the extreme biasing effect of Mann et al’s “short-centered” principal component analysis.
First, I’ll fill in some much needed context as an antidote to McIntyre and McKitrick’s misleading focus on Mann et al.’s use of principal components analysis (PCA) in data preprocessing of tree-ring proxy networks. Their problematic analysis was compounded by Wegman et al.’s refusal to even consider all subsequent peer reviewed commentary – commentary that clearly demonstrated that correction of Mann et al.’s “short-centered” PCA had minimal impact on the overall reconstruction.
Next, I’ll look at Wegman et al.’s “reproduction” of McIntyre and McKitrick’s simulation of Mann et al.’s PCA methodology, published in the pair’s 2005 Geophysical Research Letters article, Hockey sticks, principal components, and spurious significance). It turns out that the sample leading principal components (PC1s) shown in two key Wegman et al. figures were in fact rendered directly from McIntyre and McKitrick’s original archive of simulated “hockey stick” PC1s. Even worse, though, is the astonishing fact that this special collection of “hockey sticks” is not even a random sample of the 10,000 pseudo-proxy PC1s originally produced in the GRL study. Rather it expressly contains the very top 100 – one percent – having the most pronounced upward blade. Thus, McIntyre and McKitrick’s original Fig. 1-1, mechanically reproduced by Wegman et al., shows a carefully selected “sample” from the top 1% of simulated “hockey sticks.” And Wegman’s Fig. 4-4, which falsely claimed to show “hockey sticks” mined from low-order, low-autocorrelation “red noise,” contains another 12 from that same 1%!
Finally, I’ll return to the central claim of Wegman et al. – that McIntyre and McKitrick had shown that Michael Mann’s “short-centred” principal component analysis would mine “hockey sticks,” even from low-order, low-correlation “red noise” proxies . But both the source code and the hard-wired “hockey stick” figures clearly confirm what physicist David Ritson pointed out more than four years ago, namely that McIntyre and McKitrick’s “compelling” result was in fact based on a highly questionable procedure that generated null proxies with very high auto-correlation and persistence. All these facts are clear from even a cursory examination of McIntyre’s source code, demonstrating once and for all the incompetence and lack of due diligence exhibited by the Wegman report authors.
TO SAVE SPACE, I LEFT OUT THE MIDDLE OF DEEP CLIMATE'S POST; PLEASE GO TO LINK AT BOTTOM TO READ THE ENTIRE POST.
So Wegman et al.’s “compelling” demonstration is shown to be completely false; the biasing effect of “short-centered” PCA is much less evident when applied to AR1(.2), even when viewing the simulated PC1s in isolation. To show the extreme effect claimed by McIntyre, one must use an unrealistically high AR1 parameter. This is yet one more reason that the NRC’s ultimate finding on the matter, namely that “short-centered” PCA did not “unduly influence” the resulting Mann et al. reconstruction, is entirely unsurprising.
In summary, then, I have shown in Wegman et al. a misleading focus on one particular step of the Mann et al. algorithm, accompanied by a refusal to substantively consider the the peer-reviewed scientific critiques of M&M.
And to top it all, Wegman et al. flatly stated that the biasing Mann et al. algorithm would produce “hockey stick” reconstructions from low-order, low-autocorrelation red noise, while displaying a set of curves from the top 1% of “hockey sticks” produced from high-persistence random proxies. Those facts are clearly apparent from McIntyre and McKitrick’s source code, as modified and run by Wegman et al. themselves.
Make no mistake – this strikes at the heart of Wegman et al.’s findings, which held that the writing of Mann et al. was “somewhat obscure,” while McIntyre and McKitrick’s “criticisms” were found to be “valid” and their arguments “compelling.” And yet the only “compelling illustration” offered by Wegman et al. was the supposed consistent production of “hockey sticks” from low-correlation red noise by the Mann et al. algorithm. But the M&M simulation turned out to be based on nothing of the kind, and, to top it all, showed only the top 1% of simulated “hockey stick” PC1s.
So there you have it: Wegman et al.’s endorsement of McIntyre and McKitrick’s “compelling” critique rests on abysmal scholarship, characterized by deliberate exclusion of relevant scientific literature, incompetent analysis and a complete lack of due diligence. It’s high time to admit the obvious: the Wegman report should be retracted.
No comments:
Post a Comment