The Scientific Method

Aristotle was a Greek philosopher who lived from 384 BC to 322 BC. His works contain the first known formal study of logic, which he applied in many areas of life, including the field of science. Aristotle made extensive observations of natural phenomena and then applied logic to these observations in an effort to systematize them. Sometimes these logical inferences were correct, for example his deduction that the Milky Way is not shaded by the earth from illumination by the sun because the sun is too large and the stars are too distant for this to occur. Sometimes Aristotle's reasoning led to incorrect conclusions, such as his belief that the sun, stars and planets circle the earth. And occasionally Aristotle's conclusions were incorrect because his observations were not as careful as they might have been--for instance, he believed that men have more teeth than women, and that heavier objects fall faster than lighter ones.

Aristotle believed that observations coupled with reasoning could decipher the laws of the universe. He discounted experiments as artificial contrivances with little relevance to the natural world. When isolated events contradicted the laws of the universe as he understood them, they were regarded as "monsters" that could be ignored. Because Aristotle was very highly thought of as a philosopher and logician, it was regarded as a form of heresy to contradict the laws of science Aristotle had deduced from his observations. For that reason, his incorrect scientific ideas carried a great deal of weight at least until the 1500's.

In the 1500's, men such as Francis Bacon and Galileo Galilei brought changes to the study of science. Bacon rejected the idea of science by logical reasoning and syllogism. He advocated the use of observation, hypothesis and experiment leading to a gradual and systematic formulation of general axioms which could be disproven if evidence came forth to contracdict them (the scientific method, illustrated above). Galileo, as every schoolchild knows, availed himself of technology that had not been available to Aristotle. His telescope revealed that satellites orbited the planet Jupiter and that the planet Venus had phases just like the moon. While logic dictated that the earth was the center of the universe, experimental observations made by an Italian physicist indicated that this could not be the case.

When science was dominated by the application of deductive reasoning, scientific progress was slow to nonexistent. Even highly educated people believed in such things as phlogiston and spontaneous generation. Thanks to the scientific method, experiments were performed by Antoine Lavoisier, one of the men who discovered oxygen, and we now realize that burning is not a process of releasing an invisible, weightless substance called phlogiston, but a process of oxidation. Thanks to the scientific method in the hands of Louis Pasteur, we know that flasks of broth do not become cloudy by creating bacteria on their own, but that microscopic organisms can reproduce and multiply in a broth that initially appears clear.

With all of this in mind, it is surprising that some 21st century health experts wish to return to the days of science by deductive reasoning. While certain phenomena may appear to be true by anecdote or under certain conditions, without a systematic comparison of different interventions, there is no way to know for sure if eating a particular type of diet is good for weight loss, weight maintenance or (more importantly) the avoidance of the diseases of Western civilization. As Gary Taubes says at the conclusion of Good Calories, Bad Calories, "What's needed now are randomized trials that test the carbohydrate hypothesis as well as the conventional wisdom. ...it's hard to imagine that this controversy will go away if we don't do them, that we won't be arguing about the detrimental role of fats and carbohydrates in the diet twenty years from now. ...it's hard to imagine that the cost of such trials, even a dozen or a hundred of them won't ultimately be trivial compared with the societal cost."

Some investigators are doing randomized clinical trials, such as the A TO Z Weight Loss Study to compare diets such as Atkins, Ornish, Zone and a standard low-fat/high-carbohydrate diet. More of these studies need to be done, so that we can understand the specific health effects of eating various types of diets in various types of people over extended periods of time. And even in the context of low-carbing, it would also be helpful to have studies that examine the effect of eating saturated fats vs. polyunsaturated fats; eating at least 12-15 carbs' worth of vegetables per day vs. eating very few plant foods; including dairy vs. avoiding dairy in our diets; and taking various supplements vs. using no supplements. Until we have the studies to confirm or disprove our presuppostions, we are on shaky ground, just like Aristotle. Most of the time he was correct. Some of the time he was not. Without the scientific method, it's hard to know which is which.

Good Calories Bad Calories Is Not Necessarily Infallible

When Good Calories Bad Calories (abbreviated here as GCBC) was published in 2007, the low-carb community was ecstatic. Dr. Robert Atkins and the Doctors Eades had discussed the scientific basis for the low-carb lifestyle, but their writings were usually presented in the context of clinical observations. With GCBC, Gary Taubes gave low-carbers 460 pages of tightly reasoned discussion and another 113 pages listing many specific citations from the scientific literature.

For a layperson, the book was not easy to read, but with effort it was comprehensible. At last low-carbers had access to information that cast doubt on the hypothesis that excessive consumption of fat raises cholesterol levels, which in turn causes heart disease and early death. Taubes presented plausible evidence for an alternative hypothesis--that excessive carbohydrate consumption, not fat consumption, is the cause of diabetes, heart disease, hypertension and even cancer.

Since the publication of GCBC, two interesting things have happened. (A) GCBC has moved into the position of holy writ in the eyes of many low-carbers and (B) several low-carb blogs and forums have arisen to discuss the scientific and practical aspects of low-carbing.

A rereading of GCBC in 2010 shows that many of its ideas have been supported by the subsequent publication of prospective dietary studies, including Weight Loss with a Low-Carbohydrate, Mediterranean, or Low-Fat Diet, published in the New England Journal of Medicine. However, recent discussions in the blogosphere show that some statements in GCBC may need to be reconsidered.

Specifically, on page 394 of the hardbound edition of GCBC, Taubes states, "By the mid-1960s, four facts had been established beyond reasonable doubt: (1) carbohydrates are singularly responsible for prompting insulin secretion; (2) insulin is singularly responsible for inducing fat accumulation; (3) dietary carbohydrates are required for excess fat accumulation; and (4) both Type 2 diabetics and the obese have abnormally elevated levels of circulating insulin and a 'greatly exaggerated' insulin response to carbohydrates in the diet..."

Let's address these statements in order.

1. Although consumption of carbohydrates does prompt insulin secretion, it is a well-known physiological fact that consumption of proteins also prompts insulin secretion. The amount of insulin released in response to protein is about a third of that released in response to carbohydrate on a gram-for-gram basis, but the increase is still measurable. Dr. Mike Eades has an illustration of this on page 37 of the paperback edition of Protein Power. Scientific articles measuring the insulin release in response to protein can be found here and here. Insulin response to various foods in terms of 120 minute area under the curve can be found in Table 4 here.

2. Insulin release does promote the storage of fat in adipocytes, but it is not the only signaling protein that produces fat storage. Acylation Stimulating Protein (ASP) is secreted by fat cells and allows fat to be removed from chylomicrons and stored in fat cells. Acylation Stimulating Protein permits the body to store fat in the absence of insulin. The process is discussed here by Dave Dixon and here by Petro Dobromylskyj (Hyperlipid).

3. While it is difficult to accumulate excess fat in the absence of dietary carbohydrates, it is not impossible. On various discussion boards, a few zero-carbers have related anecdotal evidence that they gained weight while eating large amounts of protein and fat. From a theoretical perspecive, on pages 388-392 of GCBC Taubes goes into great detail about the necessity of glycerol phosphate for the storage of fat in adipose tissue. (Glycerol phosphate is the precursor to the molecule used as the backbone of a triglyceride, the storage form of fat.) On page 392 Taubes says, "Dietary glucose is the primary source of glycerol phosphate. The more carbohydrates consumed, the more glycerol phosphate available, and so the more fat can accumulate. For this reason alone, it may be impossible to store excess body fat without at least some carbohydrates in the diet and without the ongoing metabolism of these dietary carbohydrates to produce glucose and the necessary glycerol phosphate." This sounds logical. However, biochemists know that glycerol phosphate can readily be produced from protein via glyceroneogenesis. The absence of dietary carbohydrate in no way prevents the synthesis of triglycerides from a high-protein or even a high-fat diet.

(4) It is true that high insulin is often associated with type 2 diabetes, but it is important to remember that type 2 diabetics do not always have an excess of circulating insulin. Instead they have insulin resistance. If their body tries to control high blood glucose levels with excess insulin production by the pancreas, this can result in beta cell burnout and a patient who actually has less endogenous insulin production than a person without diabetes.

As described here the scientific method is an ongoing process. Good Calories Bad Calories is an excellent book and provides many good arguments for the low-carb lifestyle. But the scientific method requires that we keep testing and evaluating our hypotheses, and it is important to realize that not everything we read in GCBC will necessarily stand the test of time.

Scientists Behaving Badly

The premise of this blog is that the scientific method can be used to support or invalidate the tenets of the low-carb lifestyle. While science can never claim to establish the final truth of a particular hypothesis, it is the best instrument we have to approximate the truth of something that is falsifiable, that is, something that is capable of being tested by experiment or observation.

Although science is an excellent tool, we must be careful to remember that science is performed by human beings who are not perfect. Low-carbers are already aware of the problematic work of Dr. Ancel Keys. Among Dr. Keys' most important publications was the Seven Countries Study. This study helped establish the diet-heart hypothesis when it found that in seven specific countries, the cardiovascular disease rate was positively correlated with average serum cholesterol and per capita intake of saturated fatty acids. In 1957 two scientists, Jacob Yershalmy and Herman Hilleboe, noted that data were available from 22 countries, not just seven. They published a paper showing that when all 22 countries were analyzed, the cholesterol/saturated fat correlation to heart disease became much weaker, and the incidence of heart disease was more strongly related to sugar intake. Even though it seemed that Dr. Keys might have cherry picked his data, his diet-heart hypothesis has nonetheless prevailed over the years.

The science of Anthropogenic Global Warming (AGW) doesn't have much to do with low-carbing, but it does have a great deal to teach us about the practical aspects of whether to believe or disbelieve a particular scientific finding. In November 2009, a series of e-mails was made available on the internet, purporting to be from the Climate Research Unit (CRU) at the University of East Anglia in Norwich, England. As of this writing, their authenticity has not yet been denied, and these e-mails now form the heart of what has been termed Climategate.

What does Climategate have to tell us about how to evaluate scientific claims with a skeptical eye?

First, if the scientists refuse to release their raw data, it's not a good sign.

Phil Jones (head of the CRU) and Tom Wigley (University Corporation for Atmospheric Research in Boulder, Colorado) discuss here how to avoid releasing data in response to a Freedom of Information request. Dr. Jones is so averse to scrutiny of his data that he admits to clearing e-mails off his computer here and advises his colleagues to do the same here. (AR4, referenced in this link, is the Fourth Assessment Report of the UN's Intergovernmental Panel on Climate Change (IPCC), released in 2007. The AR4 allowed AGW supporters to claim a consensus in favor of anthropogenic global warming.)

Second, if the scientists select or massage their data to make it obey their hypothesis, it's a bad sign.

Dr. Jones has a problem because his data shows declining recent temperatures rather than rising ones. Here he tells three of his colleagues, "I've just completed Mike's Nature trick of adding in the real temps to each series for the last 20 years (ie from 1981 onwards) amd from 1961 for Keith's to hide the decline." Trick? Hide the decline? What might that mean?

"Mike" is Michael Mann, the creator of the Hockey Stick graph that used tree ring data to show no warming in the Medieval Warm Period, but a sudden, dramatic increase in global temperature in the late 20th century. In this article, Stephen McIntyre and Ross McKitrick show that the hockey stick graph is the result of overweighting data from American bristlecone pines and from using a non-centered principal component analysis that will almost always produce a hockey stick endpoint, even from random numbers.

"Keith" is Keith Briffa, whose tree ring data from the Yamal Peninsula of Siberia also showed a hockey stick pattern of recent global temperatures. Except that when Briffa's 12 tree cores (the red line on the graph below) are compared with 34 cores from the same area analyzed by Stephen McIntyre (the black line), the larger sample does not show the hockey stick pattern, suggesting that Briffa's 12 tree cores were unrepresentative of the local tree growth patterns and should not have been used to infer patterns of climate change for the Yamal region of Siberia, let alone for the whole planet.




Finally, if the scientists collude to allow some points of view to pass the peer review process while preventing other points of view from being expressed, it's a very bad sign.

Scientific journal editors decide which submitted papers will get reviewed, who the reviewers are, and whether the papers eventually get published. Here Tom Wigley tells Timothy Carter that they must get rid of an editor of the journal Climate Research. The man subsequently resigned. Here Tom Wigley and Michael Mann discuss a troublesome editor at Geophysical Research Letters (GRL) and whether he could be ousted because his presence may bring other AGW skeptics on board. Several months later the editor has left his post and here Michael Mann says, "The GRL leak may have been plugged up now w/ new editorial leadership there." Here Phil Jones is also having trouble with a new editor of the journal Weather, published by the Royal Meteorological Society (RMS). Dr. Jones says he has complained about the editor to the RMS chief executive, but if that doesn't work, he will not send any more papers to the RMS and will resign from the organization. When a group of scientists consciously engages in encouraging some editors and intimidating others, it's not particularly surprising if their papers tend to get published in the peer-reviewed journals while those of the scientists with opposing views do not.


Presumably scientists who hide data, who change data to fit their preconceived ideas and who conspire to see that only their data is published may nevertheless have reached correct conclusions. That would be the "fake but accurate" defense. However, it is much more likely that scientists who behave in this way have something to hide. Whenever you learn that a scientist in any field has engaged in one or more of these questionable activities, be very careful of whatever that scientist has to say.