In the latest issue of Journal of Organic Production Systems, a long-term toxicology study by the Australian scientist Judy Carman purports to show that genetically modified (GM) feed grains cause severe stomach inflammation in pigs. With help from outlets like Reuters Health, the study’s claims have been disseminated widely and have become fodder for anti-GM activists looking to scare the broader public.

Yet the study has also encountered criticism from the scientific community. At the Biofortified Blog, Anastasia Bodnar argues that study was invalid because the researchers failed to make sure that the GM and non-GM feed had similar nutrient composition. Swine health management specialist Robert Friendship has also said that the authors were incorrect to use redness as a measure of inflammation.

A number of critics found have taken issue with the paper’s use of statistics. Several people have charged the authors with “statistical fishing,” and I plan to write about that charge in the very near future. For now, I want to focus on another criticism which has arisen: that the claimed relationship between inflammation and GM-feed was based on an ill-chosen statistical test.

Source: michaelpollan.com.

What’s the single most important thing that ordinary people can do to reform their food system? In his new book, Cooked: A Natural History of Transformation, food movement leader Michael Pollan argues that it’s cooking at home. He points to the fact that the amount of time Americans spend cooking has dropped by half since the sixties, while at the same time we’ve increased our consumption of unhealthy processed foods and seen “the rise in obesity and all the chronic diseases linked to diet.” His conclusion is that the best way to address our public health crisis is for everybody to start cooking again.

More time in the kitchen might sound like drudgery to some, but Pollan argues that it shouldn’t. He cites the work of Harvard primatologist Richard Wrangham, who has hypothesized that cooking “set us apart from the apes and made us human.” Pollan argues that cooking is in our nature, so those of us who aren’t cooking are missing out. He also sees the rise in popularity of television cooking shows as evidence that “there are things about cooking we really miss.” From there, we’re to conclude that if we returned to the kitchen, we’d be happier and healthier.

In 1898, the chemist and physicist William Crookes devoted his presidential address to the British Association for the Advancement of Science to “a life and death question for generations to come.” At existing rates, he argued, the world’s wheat crop would cease to be able to feed the world’s wheat-eating people within a few decades. Crookes based his forecast on an estimate of the amount of nitrogen available for the growth of wheat crops. For centuries, farmers had observed that land would become less productive when it was planted year after year. With the advent of modern chemistry, scientists came to understand that these declining yields resulted from dwindling levels of nutrients in the soil as these substances were taken up by crops and removed from the land at harvest.

Rhizobia fix nitrogen in nodules of the roots of legumes (soybean pictured). Photo copyright International Institute of Tropical Agriculture. Used under Creative Commons License

All life requires nitrogen, which is an important component of many vital molecules, including DNA and proteins. Elemental nitrogen makes up nearly four-fifths of the earth’s atmosphere, but plants and animals can’t use nitrogen in that form. Before atmospheric nitrogen can be used by plants or animals, it must be converted to the biologically available form, or fixed. In Crookes’ day, virtually all of the biologically available nitrogen had been fixed by microorganisms known as diazotrophs, such as the bacteria of genus Rhizobia in the roots of legumes like beans, clover, and alfalfa. European and North American farmers had begun augmenting yields by importing nitrogen-rich bat guano from Peru and sodium nitrate from Chilean mines. However, these were not unlimited resources, and Crookes reasoned that unless something were to change, the lands available for wheat cultivation would cease to be able to produce enough wheat to meet world demand by the 1930s.

“We were once just as certain that the world was flat. We were wrong then, and we are wrong again.” That’s what Allan Savory had to say at TED about science’s view of desertification, a form of land degredation in which land loses its vegetation and bodies of water¹. For years, scientists have believed that overgrazing by livestock is a major cause of desertification, and Allan Savory shared this opinion as a young biologist. Now Savory argues that we can prevent desertification by grazing more animals, with their movement carefully planned to mimic nature.

TED curator Chris Anderson dubbed Savory’s talk the highlight of the year’s conference and the official TED Twitter feeds described the talk as “profound and important.” Predictably, the talk was cheered by pastured meat enthusiasts, with Michael Pollan appearing ready to recant his advice to eat “mostly plants”:

Eat MORE meat? Highlight of TED: Allan Savory on How to green the world’s deserts and reverse climate change.bit.ly/XOmROD

— Michael Pollan (@michaelpollan) March 4, 2013

Savory’s talk was also well-received in typically level-headed corners of the Twittersphere, with Discover Magazine calling it “one of the most thought-provoking” talks of the year, and Michael Shermer declaring it “moral progress in climate change.”

As C.J. Hadley’s profile in the February 2000 issue of the journal Rangelands makes clear, Savory has led an interesting life. He was born and raised in Zimbabwe (then Rhodesia), and worked in biology, military, politics, and other fields. Once a staunch opponent of grazing, he came to believe that we need more cows rather than fewer. As he told Hadley, “Severe grazing is absolutely essential to maintain biodiversity.” Overgrazing, he argues, is not the cause of desertification, arguing that grasslands have historically evolved with grazing animals. He now believes that the problem is poorly-planned grazing, and he now advocates for “holistic management and planned grazing” in order to address “all of nature’s complexity and our social, environmental, economic complexity.” According to Savory, well-planned grazing promotes plant growth, and the action of animal hooves on the soil improves the soil’s absorption of water. These ideas support the view that well-planned animal grazing can prevent and reverse desertification, thereby helping to combat climate change.

Savory fled Zimbabwe for Texas in the 1970s. In 1984 he and his wife, Jody Butterfield co-founded the non-profit Center for Holistic Management (later renamed the Allan Savory Center for Holistic Management and then the Savory Center and then Holistic Management International) to promote his approach to cattle grazing and land management. In 1992, the couple founded the Africa Centre for Holistic Management, which aimed to promote the same methods in Africa. More recently, Savory and Butterfield have left Holistic Management International to start the for-profit Savory Institute, another organization devoted to promoting Savory’s Holistic Management methods. ²

Savory’s methods have found little support from mainstream science. The same issue of Rangelands that featured Hadley’s profile of Savory also included an article by Jerry L. Holechek and others, which attempted to review the evidence for a number of Savory’s claims. Their review of studies from 13 North American sites and additional data from Africa found little evidence for any of the environmental benefits which Savory claimed for his methods. Moreover, the research consistently indicated that “hoof action from having a large number of animals on a small area for short time periods reduced rather than increased infiltration,” seemingly contradicting a key assumption of Savory’s methods.

In a letter to the editor of Rangelands that June, Savory dismissed Holechek’s findings, pointing out that the cited studies considered “short duration grazing” schemes rather than the Holistic Management methods for which Savory advocated. He explained, “The work Holechek et al. describe is unlike any range management practice I have ever advocated…In fact I have consistently stated that all grazing systems and rotations, including short duration grazing, will fail.” ³ As Savory explained in 1983, a key difference between short duration grazing and Savory’s Holistic Resource Management is that the latter method is “time-controlled,” adjusting grazing periods in accordance with the rate of plant growth.

Savory went on to describe what he identified as the only research trial ever conducted on his methods, which took place in Zimbabwe in the 1960s. The experiment, known as the Charter Grazing Trials and undertaken with Savory’s involvement, tested the claim that “we could double the stocking rate on any land under conventional management, improve the land and make more profit” with Savory’s methods. Savory reports, “The only trial ever conducted proved what I have always advocated and continue to advocate when livestock are run on any land.”

In general, it is unlikely that a single study on a few plots of land will definitively prove a statement about “any land.” Moreover, while I haven’t seen the original papers (which were published in the Zimbabwe Agricultural Journal), Holechek summarized the published work in a later issue of Rangelands, finding relatively weak support for Savory’s methods.

A further complication with the Charter Grazing Trials is that, as Savory’s letter notes, it tested “the planned grazing process then called short duration grazing, but today called Holistic Planned Grazing.” Indeed, Savory wrote in his 1983 Rangelands piece, “For those without a deep knowledge of Short Duration Grazing, I am extremely familiar with this grazing system as I developed it.”

The best I can do to make sense of all this is to put everything in the context of Savory’s explanation of his method. In his 1983 article in Rangelands, Savory explained that the best analogy for what was then called the “Savory Grazing Method” was a computer:

If you know what you want to do with the computer and you feed in the right information it will aid you in achieving the result rapidly and surely. If some factors change and you observe the changes and feed in the new information the computer will again aid you efficiently in getting to your goal.

Thus, it is not necessarily contradictory for Savory to claim to have developed short duration grazing, that short duration grazing doesn’t work, and that the methods for which he has always advocated have been proven to work. The key is that the method that would become short duration grazing was the output of Savory’s “computer” when he fed in the information that was relevant to that situation. ⁴

That said, this ought to serve as a warning against the folly of claiming a single study as proof of anything. If the Charter Grazing Trials are proof of something, why is it not short duration grazing which was validated by its results? It is, after all, the method that was employed in the study. For Savory to now say that short duration grazing is a failure, he has to concede that a single study doesn’t prove the effectiveness of a method on “any land.”

Holechek’s 2000 article also claims that Savory had “expressed doubt that holistic resource management could be validated experimentally.” While I was not able to find a precise reference for this claim, Savory did not deny it in his response, and elsewhere he has expressed some reservations about scientific testing. For instance, in Hadley’s (mostly favorable) profile, he made it clear that he saw little use for the scientific method:

You’ll find the scientific method never discovers anything. Observant, creative people make discoveries. But the scientific method protects us from cranks like me.

While the intended irony of this statement has not escaped me, it’s hard to read it without concluding that he sees the scientific method as an obstacle to progress, rather than a tool for achieving it.

That is problematic because the scientific method is what will tell us whether Holistic Management works. Savory would like us to graze more cattle to fight desertification and climate change, even as scientific evidence indicates that his “solution” will actually exacerbate these problems. If we were to employ Savory’s methods on a large scale and they were to succeed in stopping desertification, that would constitute excellent experimental validation of those methods. ⁵ If we can’t experimentally validate his methods, then we cannot know them to work in real life, either. It is thus perfectly reasonable to ask for evidence on a smaller scale before we try such a thing, particularly as existing evidence indicates that this would make the problems worse. As Chad Kruger writes, “Being ‘unconventional’ is not, in itself, a problem, but when what you are arguing for is unconventional, you’d better ‘bring data.’”

It is also important to note that for all Savory’s insistence that his methods work, it has been associated with a number of failures. For instance, Hadley mentions a test farm in Zimbabwe, which collapsed as soon as Savory fled that country. Whereas those on the farm blamed the collapse on drought, Savory blamed on their lack of proper planning in his absence.

This is typical of Savory’s response to failure. The fault never lies in his methods but in people’s implementations of them. For instance, in a 1990 paper in the journal Ecological Economics⁶, Savory explains away “15 years of frustrating and eratic [sic] results” with the admission that “we had confused the integrated approach with the holistic approach, thinking that the terms were synonymous,” emphasizing that “the breakdowns we were experiencing were not attributable to the basic concept being wrong but were always due to management–of the people and the finances.” ⁷ Even after Savory realized that “the integrated approach and the holistic approach were opposites,”⁸ he had to struggle with “understanding not only what ‘holistic’ meant, but even more difficult, how to apply such an approach in day-to-day management.” Even when his own implementation of his own ideas leads to failure, he believes that the problem isn’t that the methods were flawed but that he didn’t fully understand them.⁹ It is this kind of convoluted reasoning that allows him to claim that his methods work.

In a review of Savory’s 1988 book Holistic Resource Management, M.T. Hoffman wrote “The apparent inconsistencies and lack of definitions (eg. for concepts such as complexity, stability, resilience, diversity and production which have a number of different meanings in the ecological literature), render it frustratingly difficult to compare his [Holistic Resource Management] approach with the broader literature.” Imprecise language doesn’t just make it hard to compare Savory’s methods with the existing literature. It also makes it nearly impossible to evaluate his approach scientifically because it allows Savory to blame any failures on a misunderstanding of the method. So long as nobody can understand Savory’s ideas, those ideas can’t be tested or disproven. In this framework, problems are solved not by trying different methods but by developing a “better” understanding of the existing one. It is therefore little wonder that after members of the Department of Range and Forage Resources at the University of Natal - Pietermaritzburg–Savory’s alma mater–met with Savory, they reported that it would be “extremely difficult” to test Savory’s ideas because “it would be difficult to evaluate even a short-term research endeavour (of 5 to 10 years), as the approach taken by Holistic Management proponents would surely have changed by then, as it has in the past.”

Changing one’s mind in response to new evidence isn’t bad at all. Indeed, it’s the scientific thing to do. However, that is not the kind of change that one finds in Savory’s writings. Savory insists that he’s been correct all along and that his evolution flows naturally from an improved understanding of his own bulletproof ideas. Whereas the scientific method lends itself well to self-correction, Savory’s vague language and inconsistency will lead to trying many different things without reason to expect improvement.

Savory argued at TED that Holistic Management “offers more hope for our planet, for your children, and their children, and all of humanity.” What Savory does not tell us is that there is the distinct possibility that if we try to implement those ideas, we will fail. In this case, he will tell us that we misunderstood his ideas. How comforting it will be to know that his ideas were correct, as they always have been!

In December, TED responded to concerns that independent TEDx authorized events were “dragging the TED name through the mud” by sending a letter to “the TEDx community” warning that bad science could lead to revocation of the TEDx license. The letter also included some advice for identifying bad science. I can’t help but think that Savory’s work should have raised concerns for anybody familiar with that list. At the least, Savory’s work “has failed to convince many mainstream scientists of its truth,” much of it “is not based on experiments that can be reproduced by others,” it comes from an “overconfident fringe expert,” and it uses imprecise vocabulary to form untested theories.

Of course, TED has no contractual incentive to apply the standards it sets for TEDx organizers to its own talks. However, the letter emphasizes that “your audience’s trust is your top priority,” and I think it’s fair to ask what TED did to respect that trust in this case. Did they research the science behind Allan Savory’s ideas? Are they satisfied that his talk amounts to “good science”? If Savory’s talk had run at a TEDx event, would that event’s license have been revoked? Now that TED has reined in TEDx, perhaps its next move should be to look in the mirror.

Further reading

• Rotational Grazing on Rangelands: Reconciliation of Perception and Experimental Evidence (2008)
• Origin, Persistence, and Resolution of the Rotational Grazing Debate: Integrating Human Dimensions Into Rangeland Research (2011)
• An Evidence-Based Assessment of Prescribed Grazing Practices (2011)
• Short duration grazing and the savory grazing method in perspective (1983)
• TED Talk Teaches Us to Disparage the Desert (Added March 17, 2013 at 8:20AM)

---

1. An earlier version stated that desertification was “a form of land degradation in which land turns to desert.” Chris Clarke’s excellent takedown of Savory’s talk taught me that this wasn’t right.

Twice in the last week, Tom Philpott has mentioned what he considers to be “the most important ag research released in 2012.” That research is a comparison of three different crop rotations conducted by scientists at the Leopold Center for Sustainable Agriculture at the Marsden Farm at Iowa State University. The researchers compared a typical two-year rotation of corn and soy with a three-year rotation that incorporated a grain crop and a cover crop and a four-year rotation which also incorporated an alfalfa crop.

The results? As Philpott puts it, “The more diverse systems reduced the need for synthetic nitrogen fertilizer—an energy-intensive, water-polluting substance—by a stunning 86 percent, while maintaining yields.” Based on this finding, Philpott argues that the government should pay farmers to plant cover crops.

“If GMOs are perfectly safe, why can’t consumers know that they’re in their food?” So asked Stacy Malkan, media director for the Yes on 37 campaign, which led the unsuccessful fight to label foods containing genetically modified organisms, or GMOs, in California last fall. The insinuation – frequently made by advocates of the labeling initiative – was that the only reason that companies like Monsanto and Coca-Cola might spend millions of dollars to defeat the measure is that that these products were unsafe, and therefore needed to be hidden from consumers to protect the companies’ profits.

As I have argued, Proposition 37 was not about transparency, and that despite Michael Pollan’s insistence to the contrary, the mandatory labels only made sense as warnings. Some who share this view have argued that the battle over Proposition 37 points to a need for real transparency in the food system, and that the food industry should move toward more informative labels. For instance, UC Berkeley biologist Michael Eisen, a vocal opponent of the initiative, Tweeted in favor of a more informative labeling scheme:

.@globalecoguy i would love to see comprehensive labeling that reported on variety, ag practices used, inputs applied, etc….

— Michael Eisen (@mbeisen) November 3, 2012

I like the idea of giving more information to consumers who want it, but I’d hesitate to support mandatory comprehensive labeling, particularly for environmental concerns. There’s a good reason to exclude certain information from labels, one that has nothing to do with evil companies keeping secrets from consumers and everything to do with cost.

This Tuesday, California voters will decide the fate of Proposition 37, a ballot initiative which would require special labeling of foods produced with genetically engineered ingredients. Supporters argue that such a measure is a simple matter of transparency which is necessary so that consumers can make informed decisions.

Critics say that the proposition is intended to scare consumers rather than inform them. UC Berkeley biologist Michael Eisen wrote of the initiative’s backers, “They don’t want the ‘genetically engineered’ label to merely provide information. They want it to be a warning – the equivalent for GM food of the cancer warning on cigarette boxes.” Writing in The Los Angeles Times, self-proclaimed “health-conscious foodie” Alexandra LeTellier agreed, “GMO labels would be warnings, not just advisories.”

The initiative’s backers insist that the critics have it wrong. According to an article on the Yes on 37 campaign’s website, “No warning label would be required. Rather, the words ‘partially produced with genetic engineering’ or ‘may be partially produced with genetic engineering’ would be required on the back of the package – similar to what is now required for ingredient or allergen labeling.” Michael Pollan has likewise argued, “This is just about transparency and information. The idea that a label implies danger is false.”

In a limited sense, the supporters are right. In isolation, the label language required by Proposition 37 would be strictly factual. It’s not like the measure would require that a skull and crossbones appear on the front of packages of GM foods.

Yet labels can mean different things depending on the context. If anybody should understand this, it’s Pollan. Although I have been critical of Pollan’s In Defense of Food, this is one point he gets right in that book. Pollan argues that a fear of fat led American eaters to incorrectly conclude that fat-free and low-fat foods, such as the Snackwell’s cookie, were necessarily healthy. In this way, labels like “fat free” effectively became assurances of a food’s wholesomeness.