Bombshell: Energy Experts Call on Breakthrough Institute to Retract or Fix Its Deeply Flawed Energy Rebound Report
by climate guest bloggers Shakeb Afsah, Eric Ness and Kendyl Salcito* in a repost, January 20, 2012Summary
This follow-up note on the issue of energy efficiency and energy rebound is structured into three parts. We first discuss the shortcomings in the dataset used by Dr. Saunders for analyzing energy efficiency trend and rebound in thirty industrial sectors. Next we discuss that Breakthrough and the New Yorker’s David Owen have offered no convincing analytical evidence to show that the indirect macro level rebound will offset most of the gains from energy efficiency. Then we discuss how Prof. Pielke, Jr., has overlooked the use of unpublished grey literature by Breakthrough—an issue for which he has shown zero-tolerance in the case of IPCC. In conclusion we suggest that there should be a proper discussion to evaluate if Breakthrough should retract or revise its rebound report. Specific issues raised by Breakthrough and Prof. Pielke, Jr., in their blog and emails are addressed in the appendix.
Until the time Breakthrough and David Owen claims can be validated and peer-reviewed, they cannot be used in any meaningful form in policy discussions.
*We have benefitted from our conversations with Danny Cullenward and Jonathan Koomey of Stanford, and Skip Laitner of ACEEE. All errors and opinion expressed in this note should be attributed only to the authors and the CO2 Scorecard Group.
Inappropriate Dataset for Rebound Analysis
In our research note “Energy Efficiency is for Real, Energy Rebound a Distraction” we showed that the energy rebound estimates in Saunders (2010) are either over-estimated, as in the case of the steel sector, or wrong, as in the case of mining and electric utilities. The problems don’t stop there. In addition to weaknesses in the theoretical model and the empirical methodology, the dataset used in Saunders 2010 is itself inappropriate for analysis of energy rebound. All three essential components of his paper—the theory, the empirical econometric methodology and the data – come up short.
There is a simple reason that Saunders’ dataset is not appropriate for analysis of energy rebound and energy efficiency trends: it is a dataset on the monetary value of energy expenditure by industrial sectors at a highly aggregate level. This means that the numbers in the dataset are current values of energy cost reported in million dollars—not in BTU or MWh or Joules, the conventional measures of energy use. As a result, it is impossible to directly estimate energy efficiency, productivity or intensity in physical units.
How did Saunders convert dollars to energy efficiency? Using the principle of duality from micro-economics it is theoretically possible to convert the question of production to a question of cost—but it comes with a number of mathematical and statistical assumptions. According to the principle of duality, the production function—how factories produce their goods—is the primal (original) question, and the cost function is its reflection in the mirror—the dual, the doppelganger. All this is fine in theory, but reality tells a different story.
Assumptions Rule
Converting data on energy expenditure to estimates of energy use, rebound or efficiency is fraught with risks. With the number of estimates and assumptions it involves, it’s the mathematical equivalent of estimating the travel time from Washington, DC, to New York City in a hot air balloon in bad weather. There’s no certainty you’ll arrive at your destination, let alone on a schedule. Saunders’ expenditure data is a hot air balloon, and the accompanying assumptions provided all the weather it would take to send the analysis off course.
One key assumption underlying the application of duality is that energy price is the sole factor driving producers to adopt energy efficient technologies or production processes. Prices definitely matter, but there are many other factors that influence an industry’s technology choices, as discussed in our last week’s research note. Such considerations are excluded from Saunders’ methodology.
Additionally, Saunders suggests that production systems are “adaptable and flexible” and can seamlessly substitute energy for other factors of production. As we pointed out in the electric utility and mining sectors, once an operation starts, managers are often locked into their production processes and technologies, sometimes for decades. This reality defies the image of flexibility that Saunders describes in this video clip (at 6.45 min). Many times producers are not at liberty to change their technology every year; they can tinker at the edges but frequent technological change towards energy efficient systems is not a norm—a point also described in Greening (2000).
Finally, the dataset Saunders uses is highly aggregated for macro level analysis—the big picture assessment—but he attempts to validate it at the micro level. For example, at the 2-digit level of industrial aggregation (as in Saunders’ data), the estimates for primary metal merge the steel sector with iron, aluminum and other non-ferrous metals. Yet Saunders credits the adoption of efficient electric arc furnaces with massive energy rebound for the entire primary metals sector. It’s like using a feather to calibrate a truck scale. As our earlier research note showed, his implied energy efficiency estimate for the steel sector is more than 100% off.
We are not the first researchers to cast doubt on Saunders’ dataset. In early 2011, Skip Laitner of ACEEE, and Prof. Jonathan Koomey and doctoral student Danny Cullenward at Stanford noted the inappropriateness of Saunders’ dataset for energy rebound analysis. Prof. Koomey documented his exchange with Breakthrough on this matter in considerable depth—a note we strongly recommend for all.
The bottom line is that Saunders’s analysis is built on far too many assumptions, which put his results on thin ice. Saunders recognizes the risks associated with these assumptions and lists more than five pages of caveats in his paper. He even notes that “This impressive list of limitations should not be dismissed out of hand by rebound analysts as minor or irrelevant.” Saunders should be credited for making these assumptions transparent, and if this paper were to remain cloistered in academia, these data caveats would serve as essential input for continually improving the research. But once such an unpublished research crosses over into policy discussions, as with Breakthrough’s reports and blogs, the same data caveats should be treated like the fine print of a loan or a credit contract with Banks—the devil is in these details.
The Macro Energy Rebound Illusion
We have already shown that the direct rebound effect of energy efficiency for household, transportation, and industrial sectors is low and tolerable in the range of 0-30%. These sectors together account for most of the energy consumed at the national and global levels. On the indirect rebound effect at the macro level, Dr. Amory Lovins of the Rocky Mountain Institute and Prof. Jim Sweeney of Stanford (Koomey 2011) have already highlighted the problem of double-counting of energy use in Breakthrough’s formulation.
So why has David Owen of the New Yorker argued that “rebound” effects will offset much, most, all, or more than all energy savings from increasing end-use efficiency. Analysts have repeatedly pointed out that re-spending effect at the macro level is capped between 6-8% on average, and explained in detail in blogs at the Rocky Mountain Institute (RMI) and the Council for Foreign Relations (CFR). In fact, Michael Levi exasperatedly comments in his CFR blog post: “Schipper basically makes this point to Owen, without the numbers and without confronting this specific example; why Owen can’t see what it says about this particular case is beyond me.”
Owen explained his position in a recent interview:
“For Lee [Schipper], energy is a very small part of the economy – 6 or 8 percent. But looking at it that way understates the role of energy in what we do. If we eliminated energy consumption from the economy our energy consumption [sic] wouldn’t fall 6 or 8 percent, it would completely disappear. Increasingly we are dependent on that primary energy to power almost everything. You see it directly in your personal life when you lose your power. There’s nothing you can do. You can turn on your car to charge your cellphone, as I did, but nothing else in my life worked.”
All that is true but it is neither an evidence of large economy-wide energy rebound nor a validation of the claim that indirect rebound effects can offset the savings from energy efficiency. That removing energy from our economy will shrink it by more than 6-8% means that factors of production in our economy are interlinked perhaps serially with each other in complex ways and their co-existence is necessary for maintaining our economic output at the current equilibrium. Such a notion is captured in a 1993 paper titled “The O-Ring Theory of Economic Development” inspired by the way the space shuttle Challenger exploded due to the failure of one component—the O Ring. David Owen appears to have a similar idea about the role of energy in our economy. He is right. If there is mistake, as in the case of Challenger, and our entire energy supply system goes down, the economic impact will be much more than the 6-8% of the GDP.
But Owen’s argument is incomplete. So far it is an imaginative brainstorm, not a detailed analysis. To prove that the indirect economy-wide rebound effect is large in such a framework of production system, Owen still needs to explain the precise mechanism through which rebound effect has a multiplier effect. It will require a mathematical formulation supported by proper economic concepts and real-world evidence. Once such a mechanism is available publicly it can be reviewed, even empirically tested and there can be substantive discussions of the indirect rebound effect. Indirect macro level rebound has undergone no such process, and we encourage Breakthrough and David Owen to begin a scientific validation.
Dr. Jonathan Koomey has urged the same. He writes: “The normal burden of proof is on those advocating the existence of some unexpected and novel effect to show the underlying causal mechanisms that lead to that result, so the assumptions can be peer-reviewed.” Until the time Breakthrough and David Owen claims can be validated and peer-reviewed, they cannot be used in any meaningful form in policy discussions.
The Honest Broker’s Blind Spot
Our initial research note focused on substantive problems in Breakthrough’s rebound report. Prof. Pielke, Jr., reminded us of the importance of protocol and standards for using unpublished research, however, which forced us to closely examine Breakthrough’s own formal reports. Prof. Pielke, Jr., in particular has shown zero-tolerance when the Inter-Governmental Panel for Climate Change (IPCC) has used unpublished or non-peer reviewed papers in reports. In his blog post from December 22, 2009, titled “Peer Review in the IPCC” Prof. Pielke, Jr., writes:
“This situation highlights the problem of the “laundering of grey literature” associated with IPCC reports, which occurs when an analysis or claim occurs outside the peer review literature and is subsequently cited in the assessment report.”
On January 21, 2010, he told the New York Times that “….he was concerned that, in this case, “a non-peer reviewed source [was] elevated to a finding by the IPCC...” Similarly on July 12, 2010, once again in his blog he wrote:
“We can conclude unambiguously that the citation of Rowell and Moore by the IPCC was improper as it was not only “grey literature,” but also devoid of scientific support for the claims that it advanced that were repeated in the IPCC.”
These are good principles regarding the use of grey literature. In a quick tabulation of more 1,300 of Prof. Pielke Jr’s blogs between June 13, 2009, and January 16, 2012, we found that nearly 20% of his posts use the terms or the variants on peer-review, unpublished and published.
Yet when it comes to using unpublished research in the energy rebound report by the Breakthrough Institute, where he is a fellow, he turns a blind eye. In Breakthrough’s “Energy Emergence: Rebound & Backfire as Emergent Phenomena” the authors boast a 96-article bibliography. Remarkably, some of the most salient points in the Breakthrough report cite Saunders (2010) as the source. Though Saunders is a published author, his 2010 piece has not, to date, been picked up by a peer-reviewed journal.
Conclusions
We therefore propose that Breakthrough should consider retracting or revising its rebound report. The same applies to the EU Environmental Commission report. Further in the interest of consistency, Breakthrough should adopt and follow the same standard on grey literature as their colleague Prof. Pielke, Jr., requires of the IPCC and other such organizations.
Appendix: Specific Response to Breakthrough and Prof. Pielke, Jr.
Our research note on energy efficiency and rebound, released on January 12, 2012, generated strong endorsements of our key findings and policy recommendation: rebound or not, energy efficiency works. Criticism came in the form of one blog post by Prof. Pielke, Jr., and several emails from the Breakthrough team, in which they expressed disagreement with our analysis. There was no direct or specific disagreement with our numbers or statistics that showed that Dr. Saunders’ estimates of energy rebound was either over-estimated or wrong, Breakthrough and Prof. Pielke, Jr., claimed that (1) we mischaracterize their position on energy efficiency and (2) we don’t understand the difference between energy efficiency and energy intensity. In a response through email to Breakthrough on January 12, 2012, we stated that:
“In our view, BTI is lukewarm at best on energy efficiency and negative to it at worst, and BTI’s emphatic case for rebound and backfire has a clear and logical policy implication: efficiency needn’t be prioritized. We stand by our analysis and interpretation based on our research.”
We think it is important to demonstrate the source of our perception, which is shared by others. While Breakthrough occasionally suggests that energy efficiency is needed, their core message remains that energy rebound under-cuts the savings from energy efficiency enough to fully neutralize it or even increase it.
From the very first paragraph from their flagship report “Energy Emergence: Rebound & Backfire as Emergent Phenomena,” Breakthrough casts doubt on the energy efficiency policy stance of the IEA, IPCC, experts like Dr. Lovins and organizations like Mckinsey.
In the second paragraph of their report Breakthrough writes:
“Economists, however, have long observed that increasing the efficient production and consumption of energy drives a rebound in demand for energy and energy services, potentially resulting in greater, not less, consumption of energy.”
The anti-energy efficiency tone continues in the FAQ section of their website, where they write: “…rebound effects mean that for every two steps forward we take in energy savings through efficiency, rebound effects take us one (and sometimes more) steps backwards. We may still move forward, but not as much as we initially expected.” In the deck of Breakthrough’s PowerPoint slides accompanying this report, one slide explicitly questions the energy efficiency estimate of the IEA.
Breakthrough attempts to counter our characterization of their position on energy efficiency by pointing out such comments as:
Q: Are you saying energy efficiency is a waste of time then? Are you arguing against pursuing efficiency?
A: Most certainly not! Truly cost-effective energy efficiency improvements make great economic sense and improved energy efficiency may be a key determinant of future economic welfare. In this sense, it may also contribute indirectly to climate mitigation and decarbonization objectives.”
In our view these are feeble statements supporting only the partial benefits of energy efficiency—improvement in economic welfare, and not how energy efficiency cuts energy use. If Breakthrough is truly supportive of energy efficiency it must retract its emphasis on high energy rebound and backfire, and embrace energy efficiency as a core policy tool for climate management as experts like Amory Lovins and Skip Laitner or organizations like the IEA, IPCC, McKinsey and many others have stated. It is not enough for Breakthrough to be on both sides of energy efficiency—first downplay it using the case of high energy rebound and then try to embrace it by making a few token statements of support.
Energy Efficiency and Energy Intensity
Regarding the issue of energy efficiency and energy intensity, we used energy intensity measured in physical units at the same level of industry aggregate as Harry Saunders. We compared annual trends, as did Saunders, to ensure that our use of energy intensity was comparable to his estimate of aggregate energy efficiency trends. From our perspective, further discussion of differences between energy intensity, energy efficiency and energy productivity is a moot point. If readers wish to have further clarification, we encourage them to write us.
No comments:
Post a Comment