Thoughts on the China Study

There are about 2000 counties in China. In the 1980’s, Cornell University did a large ecological study in 65 of them and published the data as something called the China Study. The study measured 367 variables in about 6500 adults. It captured data on diet, lifestyle and disease and included analyses of blood and urine samples. The individual results were grouped geographically, by county, producing a data set with 65 (or fewer) observations for each variable that was measured.

T. Colin Campbell, Ph.D. was a researcher in this study. In 2005 he published a book, The China Study: Startling Implications for Diet, Weight Loss, and Long-Term Health, using data collected from the China Study, from his own published research and from several other sources. When all of this information was considered together, on page 7 of the 2004 edition of the book Dr. Campbell concluded that,

People who ate the most animal-based foods got the most chronic disease. Even relatively small intakes of animal-based food were associated with adverse effects. People who ate the most plant-based foods were the healthiest and tended to avoid chronic disease.

In light of what low-carbers know personally and know from the scientific literature about the benefits of eating animal-based foods, Dr. Campbell’s conclusion is quite surprising. Vegans and vegetarians commonly use The China Study as proof that their food choices are scientifically superior to those that incorporate animal products. (Take a look at the comments on Amazon.com for just a few examples.) What’s a low-carber to think?

In 2005 Chris Masterjohn wrote a critique of The China Study. Masterjohn pointed out that the data showed that intake of animal protein did not correlate with mortality for all cancers. Although Campbell had tried to connect animal protein intake to cancer mortality through a set of six biomarkers like plasma copper and urea nitrogen, the relationships between animal protein intake, the biomarkers and the eventual deaths from cancer were poorly documented. Masterjohn also showed that Campbell had taken his own research on the tumor-promoting activity of casein in cancer-prone rats to make the astounding statement on page 104 of the book that “casein, and very likely all animal proteins, may be the most relevant cancer-causing substances that we consume.” This seems to be a bit of a logical stretch.

The discussion lay more-or-less dormant until 2010 when Denise Minger, a 23 year old English major and self-described data junkie, happened upon the raw China Study data and wrote a lengthy description of her criticisms of the book. In a 2001 symposium Dr. Campbell had summed up some of the findings of the China Study and a subsequent China Study II. He said, “Plasma cholesterol in the 90-170 milligrams per deciliter range is positively associated with most cancer mortality rates. Plasma cholesterol is positively associated with animal protein intake and inversely associated with plant protein intake.” After spending 1 ½ months of working with the raw data from the China Study, Ms. Minger found that the data in fact showed no statistically significant relationship between the intake of animal protein and cancer. (It also showed no statistically significant relationship between the intake of plant protein and cancer.)

So, how could Dr. Campbell describe a positive correlation between increased intake of animal protein and cholesterol and between increased cholesterol and cancer, while the raw data showed that there was no one-to-one relationship between intake of animal protein and cancer? The answer: confounding factors. Schistosomiasis is a common disease in China. It is caused by a worm not normally found in plant-based food nor in animal-based food but in contaminated water. Ms. Minger found that as schistosomiasis increases, plasma cholesterol increases significantly. (This may be the result of negative effects of schistosomiasis on normal liver function.) As schistosomiasis increases, the rate of colorectal cancer also increases significantly.

In other words, in counties where schistosomiasis was present, one would expect that people who had high cholesterol would also tend to have more colorectal cancer. Hence the presumed relationship between high cholesterol and cancer mortality in China would actually reflect a factor that had nothing to do with diet. And when Chinese counties with zero schistosomiasis infection are compared with respect to the relationship between total cholesterol and the rate of mortality from colorectal cancer, the correlation between the two variables disappears. In other words, Dr. Campbell’s reasoning that eating animal protein is associated with high cholesterol is in turn associated with increased cancer mortality is invalid. The data was presented in a way that it implied a relationship, but the relationship disappears with a more detailed analysis.

From there, Ms. Minger went on to dismantle one after another of Dr. Campbell’s assertions regarding the use of animal-based foods and damage to health. Other laypeople with statistical experience (here and here) also did their own data analysis and reinforced her conclusions.

Dr. Campbell responded to Ms. Minger’s criticisms here. His main objections seemed to be that she used adjectives (!), that she used univariate correlations (so did he) and that she was probably unable to have made her analyses without outside help.

Ms. Minger’s very detailed response to Dr. Campbell is here.

If you have time to read all of these links, they provide fascinating insight into the analysis and potential applications of an important ecological study. If you don’t, the bottom line is this: when you encounter a scientific study that seems to contradict everything you know about a particular subject, be sure to take a very careful look at the data to see if it might have been cherry-picked, over-interpreted, or analyzed without reference to potential confounding factors. If anything like this has happened, the conclusions of the investigators may not necessarily reflect what the data actually shows.