Another one bites the dust, and another one gone... Bob Ward takes down Richard Tol -- guess for whom the bell tolls?

Errors in estimates of the aggregate economic impacts of climate change

by Bob Ward, Grantham Research Institute on Climate Change and the Environment, April 2, 2014

With the publication this week of the contribution of Working Group II of the Intergovernmental Panel on Climate Change to the Fifth Assessment Report, I can reveal details of my ongoing struggle to correct errors in the report relating to the work of Richard Tol, a professor of economics at the University of Sussex. I have, so far, only been partially successful.

It was last October when I first started to closely examine Professor Tol’s work after it was cited by Viscount Ridley, a fellow member of the Academic Advisory Council of Lord Lawson’s Global Warming Policy Foundation, in a front-page article for The Spectator under the headline: ‘Why climate change is good for the world’.

Viscount Ridley drew primarily on an academic paper by Professor Tol which was published in the Journal of Economic Perspectives. The study drew together a number of estimates of the aggregate economic impacts of climate change and concluded that global warming of up to 2.2 °C would have a net beneficial impact for the world.

However, when I examined Professor Tol’s paper in detail, I discovered that he had made a number of errors, wrongly plotting studies which had found net negative impacts as if they were positive benefits. Of the 14 data listed in Table 1 and plotted in Figure 1 of the paper, at least four were wrong.

Taking into account all of these mistakes, there was only one study that showed significant positive effects from global warming. That single analysis had been published by Professor Tol himself in 2002 in the journal Environmental and Resource Economics, which concluded that warming of 1 °C would lead to benefits equivalent to 2.3% of global wealth. However, the paper also pointed out, on pages 63-64, that many impacts of climate change had been omitted from the study, including shifts in extreme weather and effects on amenity, recreation, tourism, fisheries, construction, transport, and energy supply.

I exchanged e-mail messages with Professor Tol to confirm that his 2009 paper contained mistakes, but he refused to give any undertaking to write to the journal to correct them.

I then decided to check some of the other analyses that Professor Tol had published on the economic impacts. Sure enough, I found some of the same mistakes in a couple of his papers published in 2012 in Environmental and Resource Economics  and in 2013 in the Journal of Economic Dynamics & Control.

In his 2013 paper, Professor Tol made four errors in his representation of the 17 data used in Figure 1 and Table 1. As in his earlier papers, he mistakenly plotted a 2006 study by Chris Hope as finding a net benefit equivalent to +0.9% of global wealth from a warming of 2.5 °C. In fact, it was still only the 2002 paper in Professor Tol’s dataset that showed any significant net benefits. Nevertheless, Professor Tol claimed in his 2013 paper that “the initial benefits of a modest increase in temperature are probably positive.”

Shortly after these discoveries, I found a leaked version of the final draft of the IPCC report which had been posted on a blog for climate change ‘sceptics.’ Although I am a registered reviewer on the report, I had not seen the final version because it was distributed only to governments and to authors.

I was stunned when I looked at Chapter 10 on ‘Key Economic Sectors and Services’ on which Professor Tol was one of the Coordinating Lead Authors. A section had been inserted on ‘Aggregate impacts’ which was based almost entirely on Professor Tol’s 2013 paper. The Chapter also included a new table and graph which were based on Figure 1 and Table 1 from his 2013 paper.

None of this material had been included in the Second Order Draft of the report (a copy of which was also leaked to a blog for climate change ‘sceptics’ that had been made available to reviewers, including me).

When I checked the 20 data in Table 10.B.1 and Figure 10-1, they both contained at least three errors, two of which had appeared in the 2013 paper. However, the data point attributed to Hope (2006) had been changed and was listed as -0.9% in Table 10.B.1, compared with +0.9% in the 2013 paper. Nevertheless, even though only 1 of the 20 data in Table 10.B.1 and Figure 10-1 showed any significant net benefits, the text in Section 10.9.2 of Chapter 10 stated: “Estimates agree on the size of the impact (small relative to economic growth) but disagree on the sign.”

I was not sure that governments reviewing the draft report would identify the mistakes, so in January I wrote to Professor Tol and other members of the Working Group to warn them of the problem.

I received a polite acknowledgement from Professor Doug Arent, who was the other Coordinating Lead Author with Professor Tol on Chapter 10.

Professor Tol’s reaction was rather less polite. He did not respond to me, but instead decided to leak onto his blog what he claimed was a corrected version of the graph to be included in the report, accompanied by disparaging comments against me for having pointed out his errors. He also circulated on Twitter a mocked up picture of me with the hairstyle of Irish pop music twins Jedward.

Given that Professor Tol seemed determined not to correct his papers, I also wrote to the editors of the three journals in which the flawed papers had been published.

I drew the attention of each editor to the problems. I also noted that many of the data that had been plotted by Professor Tol were aggregations that he had made of other authors’ research findings. For instance, 8 of the 17 data used in the 2013 paper had been aggregated by Professor Tol. I suggested to the editors that, given the other mistakes, Professor Tol should make available the details of his calculations so that they might also be verified.

Disappointingly, none of the journals has so far secured an agreement from Professor Tol to make his calculations available, which means that a number of the data included in Chapter 10 of the IPCC report remain unverifiable.

For this reason, I remain concerned about the following statement from the Summary for Policymakers from the report: “the incomplete estimates of global annual economic losses for additional temperature increases of ~2 °C are between 0.2 and 2% of income (±1 standard deviation around the mean).” These figures are drawn entirely from Professor Tol’s 2013 paper, and without independent verification of the data currently being possible, I do not regard them to have been proven robust.

Furthermore, the version of Chapter 10 that has been published on the IPCC’s website is the draft that was distributed to governments in October 2013, and still contains at least three erroneous data points in Table 10.B.1 and Figure 10-1. The text of Section 10.9.2 remains a highly misleading description of the data: “Estimates agree on the size of the impact (small relative to economic growth) but disagree on the sign.”

I will continue my efforts to have the errors in Professor Tol’s work corrected, as a service to researchers, policy-makers and the public.

Bob Ward is policy and communications at the ESRC Centre for Climate Change Economics and Policy and the Grantham Research Institute on Climate Change and the Environment at London School of Economics and Political Science.

http://www.lse.ac.uk/GranthamInstitute/Media/Commentary/2014/March/Errors-in-estimates-of-the-aggregate-economic-impacts-of-climate-change.aspx

One comment:

Bernard J. said...

Excellent work Bob - there will be many people closely following the outcome of this issue.

I'd for one would be very interested to see some greater explanation of how Tol derives his numbers. In brief encounters with him in the blogosphere he has demonstrated an underwhelming understanding of the functioning of ecosystems, and of the value of such functioning and of the biodiversity contained therein. And when I say "value" I mean value not only to current and near-future generations of Westerners, but to non-Western societies, to generations a century and more hence, and to the millions of other species with whom we share the planet.

Any economic analysis is effectively meaningless if it does not seriously and comprehensively account for the obvious as well as the subtle global ecological impacts of warming over the various increments of predicted temperature increase. Many of the impacts that will manifest with warming will be emergent - predicted by biologists in some instances and missed in others - and some of the most serious consequences are likely to occur in ways that were either not forseen, or were effectively ignored by everyone other than the ecologists and other scientific professionals who warn of them in the first place.

This matter is a weak link in the IPCC's reporting chain, and Tol is painting over the rust.