It should go without saying that writing an e-mail that includes Freedom of Information in the subject line and advises other people to delete e-mails is an act of amazing boneheadedness. At the very least, it stinks to high heaven. Of all the hacked e-mails I've been able to review so far, this one strikes me as the most damning, coming, as it does, in the context of others that make it clear that Jones was dead set on resisting Freedom of Information requests.

So why would he do such a thing?

According to a report by Olive Heffernan in Nature Magazine this past August, the story starts, more or less, with the publication of the original hockey stick paper in Nature in 1998. From early on, two Canadians, Ross McKittrick and Steve McIntyre, attacked Mann's conclusions and questioned his data. In the political fight over  climate change, their attacks received quite a bit of publicity, particularly from Republican legislators in the U.S. determined to stop any movement towards action on climate change.

From Nature:

McIntyre, who runs the Climate Audit blog, is best known for questioning the validity of the statistical analyses used to create the "hockey stick" graph...

More recently, McIntyre has turned his attention to criticizing the quality of global temperature data held by institutes such as NASA's Goddard Institute of Space Studies. Several organizations worldwide collect and report global average temperature data for each month. Of these, a temperature data set held jointly by the Climatic Research Unit (CRU) at the University of East Anglia and the UK Met Office's Hadley Centre in Exeter, known as HadCRU, extends back the farthest, beginning in 1850.

Since 2002, McIntyre has repeatedly asked Phil Jones, director of CRU, for access to the HadCRU data. Although the data are made available in a processed gridded format that shows the global temperature trend, the raw station data are currently restricted to academics. While Jones has made data available to some academics, he has refused to supply McIntyre with the data. Between 24 July and 29 July of this year, CRU received 58 freedom of information act requests from McIntyre and people affiliated with Climate Audit. In the past month, the UK Met Office, which receives a cleaned-up version of the raw data from CRU, has received ten requests of its own.

58 FOI requests in five days!

So why won't CRU comply?

According to Heffernan:

Jones says that he tried to help when he first received data requests from McIntyre back in 2002, but says that he soon became inundated with requests that he could not fulfill, or that he did not have the time to respond to. He says that, in some cases, he simply couldn't hand over entire data sets because of long-standing confidentiality agreements with other nations that restrict their use.

Gavin Schmidt, a climate scientist who helps run the RealClimate blog and is closely affiliated with many of the key players in this story, amplified this line of defense in one of his comments responding to the hacked e-mails contretemps here:

Because, as [Jones] has explained frequently, that in order to get the maximum amount of data available they gave assurances and signed memoranda with many national weather services not to distribute raw data that the NWS's would rather sell.

This argument may or not be true -- there appears to be some confusion over whether or not written copies of these confidentiality agreements exist. Insofar as I have been able to tell, the FOI requests were denied, so perhaps there was some legal basis for doing so.

But there's a much more personal and political struggle going on here.

Bluntly put, the climate scientists who have devoted their careers to proving global warming is happening do not believe that Steve McIntyre is a legitimate scientist whose real goal is the advancement of climate change science. They believe his primary goal is to undermine their work by any means necessary, and that any data they give him will be misused, abused, and ultimately become political fodder for the conservative forces who are fighting any efforts to do anything about climate change.

You can't put it more clearly than does Thomas Karl, the director of the National Oceanic and Atmospheric Administration's National Climatic Data Center, in one of the hacked e-mails.

We should be able to conduct our scientific research without constant fear of an "audit" by Steven McIntyre; without having to weigh every word we write in every email we send to our scientific colleagues. In my opinion, Steven McIntyre is the self-appointed Joe McCarthy of climate science. I am unwilling to submit to this McCarthy-style investigation of my scientific research. As you know, I have refused to send McIntyre the "derived" model data he requests, since all of the primary model data necessary to replicate our results are freely available to him. I will continue to refuse such data requests in the future. Nor will I provide McIntyre with computer programs, email correspondence, etc. I feel very strongly about these issues. We should not be coerced by the scientific equivalent of a playground bully. I will be consulting LLNL's Legal Affairs Office in order to determine how the DOE and LLNL should respond to any FOI requests that we receive from McIntyre.

From the perspective of the climate scientists involved, it seems clear that they did not trust McIntyre, did not feel that his FOI requests were legitimate scientific inquiry, and were determined to do whatever possible to resist him.

Does that exculpate them? Absolutely not. Does it explain why Phil Jones thought that private e-mails from climate researchers discussing the Fourth Assessment Report of the IPCC should be deleted? Nope, not at all. Does it demonstrate that scientific progress, despite supposedly being based on the accumulation of data and the testing of theories, can be a messy, messy business, full of personal intrigue and antipathies? Absolutely yes.

To climate skeptics, the e-mails prove that global warming is a conspiracy theory. At Wonk Room, Brad Johnson rounded up the politicized reaction:

• If you own any shares in alternative energy companies I should start dumping them NOW," says the Telegraph's James Delingpole.
• Hot Air's Ed Morrissey claims the emails discuss "repetitive, false data of higher temperatures."
• The National Review's Chris Horner salivates, "The blue-dress moment may have arrived."
• "The crimes revealed in the e-mails promise to be the global warming scandal of the century," blares Michelle Malkin.
• The Australia Herald-Sun's Andrew Bolt claims the emails are "proof of a conspiracy which is one of the largest, most extraordinary and most disgraceful in modern [sic] science."

RealClimate, a blog maintained by real climate scientists, is busy doing damage control. This story will no doubt rage for weeks, so I'm just going to pick one example of the back and forth before trying to take some time to go deeper, if merited.

Here's an e-mail that has gotten particular attention, with the supposedly damning language bolded:

Dear Ray, Mike and Malcolm,

Once Tim's got a diagram here we'll send that either later today or first thing tomorrow.

I've just completed Mike's Nature trick of adding in the real temps to each series for the last 20 years (ie from 1981 onwards) amd from 1961 for Keith's to hide the decline. Mike's series got the annual land and marine values while the other two got April-Sept for NH land N of 20N. The latter two are real for 1999, while the estimate for 1999 for NH combined is +0.44C wrt 61-90. The Global estimate for 1999 with data through Oct is +0.35C cf. 0.57 for 1998.

Thanks for the comments, Ray.

Cheers, Phil

Here's RealClimate's explanation:

The paper in question is the Mann, Bradley and Hughes (1998) Nature paper on the original multiproxy temperature reconstruction, and the "trick" is just to plot the instrumental records along with reconstruction so that the context of the recent warming is clear. Scientists often use the term "trick" to refer to a "a good way to deal with a problem", rather than something that is "secret", and so there is nothing problematic in this at all. As for the "decline," it is well known that Keith Briffa's maximum latewood tree ring density proxy diverges from the temperature records after 1960 (this is more commonly known as the "divergence problem" -- see e.g. the recent discussion in this paper) and has been discussed in the literature since Briffa et al in Nature in 1998 (Nature, 391, 678-682). Those authors have always recommend not using the post 1960 part of their reconstruction, and so while "hiding" is probably a poor choice of words (since it is "hidden" in plain sight), not using the data in the plot is completely appropriate, as is further research to understand why this happens.

So what's going on here? Put aside the question of whether the words "trick" or "hide" have nefarious or innocuous meanings. The scientific problem is that in attempting to reconstruct temperatures in the past, climate scientists are often faced with the problem that there were no humans standing around holding thermometers and writing down temperatures. So scientists use "proxies" -- tree rings, or ice cores, or fossilized clams, or lake pollen trapped in sediment. The "divergence problem" referred to above references a case where in one particular instance, tree ring variations in density did not match actual recorded temperatures after 1960.

That poses a conundrum, although not one that throws the entire science of multiproxy paleoclimate reconstruction into doubt.  More importantly, t the divergence problem, as RealClimate notes, is not a secret. It's exactly the kind of thing that climate scientists feast on. Such problems are discussed and debated every day by climate scientists (and every other kind of scientist.) The great thing about science is that the process of gathering  more data, improving models and theories is infinitely ongoing, and working out how to handle puzzlers like this, both in private e-mails and in the peer-reviewed literature, is what scientists live for.

Overall, the more data we have, the more clear it has become to the vast majority of scientists working in this field that the earth has gotten significantly hotter at an alarming rate in the last century, most likely due to increased concentrations of greenhouse gases in the atmosphere. And if there really is a smoking gun in the hacked e-mails that convicts scientists of fraudulent behavior or faking data, well, let's hear it. I'm not convinced by the above example. But if enough scientists are, that's a different matter.

In an opinion piece, also in the Journal, Michael Corkery approves of the basic principle:

This is how it is supposed to work. Rather than the federal government dictating what a company should pay its employees, shareholders are having their say.

Ever since Goldman Sachs' 1999 IPO, I've been wondering how the investment bank's big-money culture would square with the trope that a public company's first responsibility is to its shareholders. Up until now, Goldman's shareholders have accepted the basic bargain -- Goldman's stock returns far outpace the market. This success is supposedly attributable to Goldman having the smartest employees. And so the argument goes: If you don't pay the smartest employees an average of over $700,000 per person per year, they will skedaddle.

But it seems as if what looks to be "the biggest employee payout in the firm's 140-year history" is inciting some grumbling even among those who typically could not care less about outsize Wall Street compensation levels.

Corkery is skeptical that the big institutional investors will make good on the only threat they realistically have, which is to sell their stock. But there's an interesting subtext to the whole question of shareholder rights that is ironically highlighted by the Goldman dilemma.

Oftentimes, when we hear that a public company's first responsibility is to its shareholders, this is used as an excuse to shaft workers. What's the easiest way to cut costs and get a quick stock price boost? Lay off employees! It's one of the harshest truths of capitalism -- the people who do the actual work generally get the smallest piece of the pie and are most vulnerable to economic downshifts. The principle that shareholders should come first has propelled countless mergers and acquisitions with disastrous results for the employees of the companies that get sliced and diced in Wall Street's endless parlor games.

For Goldman critics, it might be satisfying to see big shareholders muttering grimly about the impropriety of massive bonuses. It also could be rewarding to see shareholders exert more force in restricting the runaway compensation of top executives, although I'm not going to hold my breath waiting for such "say-on-pay" ideals to ever become common practice. But I'd caution against seeing this episode of dissatisfaction as some great triumph of shareholder democracy over the lords of Wall Street. What the investors really want is more money for themselves -- a fat dividend payout, or higher earnings per share that would be reflected in a higher stock price.

And what that usually means is: Screw the worker. In this case, we're all OK with it, because these are Goldman employees, and everybody is mad at Goldman. But as a general rule, keeping stock prices high by keeping the clamps tightly down on workers is not ideal, from an egalitarian point of view.

Andrew,

I work for POET and your post was a nice surprise when I discovered it this morning. There are a few items that I'd like to respond to.

First, on the willingness of farmers to collect cobs. We are getting a good response from the farmers we've dealt with around our first plant in Emmetsburg. They are the ones we've laid out the complete case to: what we're paying (the $2.35 figure we released assumes $55 per ton), what the USDA adds through their Biomass Crop Assistance Program ($45 per ton) and the financial assistance we provide in purchasing equipment. When they see the equipment payback of less than two years and a nice profit per acre, they tend to embrace it.

Also, we are studying stover removal very carefully with Iowa State University. We need the land to be productive long-term for these facilities to be successful. After the first year of study, we've found that removing cobs and a small portion of husk and leaves does not adversely impact soil quality and there is no need to add additional fertilizer. This is a similar finding to many other studies on crop residue removal. We are continuing to study this with them.

Thanks again for the interest in our process. I invite you to stay tuned as we continue to improve it.

Schock also kindly provided a link to a POET press release discussing the preliminary results of Iowa State's research on the impacts of corn stover removal on soil fertility. From which I learned, inadvertently, that POET was funding the very same research I had linked to in my post yesterday.

I have no reason to doubt POET's forthrightness, and I take very seriously Robert Rapier's evaluation that "they have done a good job" on the cellulosic ethanol front. But I find it deliciously ironic that one of the top five results returned by Google for the search terms "corn stover removal soil fertility" turns out to be research paid for the company planning to do the removal.

Other studies have shown that "indiscriminate stover removal' does have a significant effect on soil fertility. Clearly, Schock is telling us that POET plans to be a discriminating stover remover. But when market forces comes into play without adequate government supervision, the line between discriminating and reckless abuse gets awful murky.

I think Ollie's got his history a little mixed up. If I recall, Republicans, following the advice of consultant Frank Luntz, as formulated in his famous 2002 memo, started using the words "climate change" instead of "global warming" because global warming was "too frightening."

From an account in the UK Guardian:

The phrase "global warming" should be abandoned in favor of "climate change", Mr Luntz says, and the party should describe its policies as "conservationist" instead of "environmentalist", because "most people" think environmentalists are "extremists" who indulge in "some pretty bizarre behavior... that turns off many voters".

I guess one example of that bizarre behavior was a little semiotic jujitsu, in which environmentalists embraced the climate change name switch, but tacked on catastrophic to get the original point across.