Diet soda and sweeteners alter gut bacteria, contributing to obesity? Not so fast.

Screen Shot 2014-09-22 at 10.13.06 AM

A study in Nature and a host of derivative pieces across the Internet have created somewhat of a niche market for making it once again fashionable to avoid artificial sweeteners. The article claims that diet sodas may alter gut microbes in some people in a way that determines whether saccharin triggers glucose intolerance, which could increases the risk of metabolic diseases such as Type 2 diabetes.

While indeed it may be true that gut microbiota can be influenced and adapt to large doses of artificial sweeteners, this is unsurprising – and in fact to have suspected otherwise would be surprising. The gut microbiome rapidly adapts to changes in the human diet, whether it’s a change from drinking water to drinking soda (diet or sweetened with sugar), or a vegetarian switching to a meat-based diet (or vice versa); the adaptability of the gut microbiota is what ensures their successful survival and proliferation.

In fact, researchers determined that giving test subjects meat “rapidly and reproducibly alters the human gut microbiome.” So a study suggesting that high doses of artificial sweeteners wouldn’t have an impact on microbes shouldn’t be surprising. Part of the reason this becomes a significant result is that the measured scope of the study is so small (framed with such specificity): For example, using high-resolution analytical equipment on surface cells in the human tongue while test subjects were given glasses of ice water to drink would show that cell osmolality is dramatically altered, making the extracellular space necessarily hypotonic, which stresses the cells; in addition, the low-temperature water creates a significant cascade of cellular communication by cell-signaling done by certain proteins (such as cytokines), and depending on the integrity of certain cells across the tongue surface lamina, the exposure to the cold water may lead to their death.

The practical result of all this is that our tongue feels cold and our thirst is quenched. Which brings up the next important factor of interpreting scientific results: Just because it can be measured doesn’t mean that it matters. This becomes more-and-more true as our analytical equipment and technologies become ever more sensitive: We can now discriminate certain lab measurements to parts-per-trillion or parts-per-quadrillion. Just because there may be a detectable trace of something doesn’t mean that it’s at all important.

For example, the xx guidance for the ability to be branded ‘Gluten Free’ is 20 parts-per-million (ppm). While this is relatively low, it’s not zero. But it is low enough to prevent measurable immunogenic response in a vast majority of subjects. So for example, in a separate study, levels of HbA1C for very heavy users of artificial sweeteners were associated with very slight increases in HbA1C levels. Given the sample size used and the precision of the measurement, this ‘increase’ may be a spurious signal, and not actually be indicative of an actual change. And besides, is a change in HbA1C (especially a miniscule change) actually indicative of anything?

The Mayo Medical Laboratories have published a small analysis showing a correlation between HbA1C levels and estimated average glucose. However, the Mayo Clinic even mentions that there is a great deal of variance in the measurement (analytical issues), an ‘adequate cutoff’ [for determining significance] has not been determined, some conditions (uremia, transfusion, and anemia) can adversely affect it (and weren’t controlled for in the above study), and only a limited number of studies have been done on HbA1C. So the HbA1C measurement is not a validated or reliable measure to make any determinations from. Add to this that the causal factors for Type 1 and Type 2 diabetes are different, and the HbA1C correlation does not distinguish, making its power for interpolating differences even lower.

Most concerning and insidious of all is that between the unreliable HbA1C proxy for diabetes, an extraordinarily small sample size was used (seven participants). The participants were given the equivalent of about 10-12 packets of artificial sweetener and about half (four out of the seven) had changes in blood glucose. So-called ‘stable’ blood sugar levels can fall around 10-15% of baseline normally with no adverse factors occurring externally – so associating disturbances in blood sugar (in only a handful of subjects) with dosing of artificial sweetener is a stretch indeed, especially at this point. It’s much more likely to be systematic error from the analysis, and the authors trying to find patterns in their data where none exist.


Another interesting argument that’s always used by those opponents of artificial sweeteners (for example diet soda) is that they contribute to such problems as fibromyalgia, lupus, brain tumors, headaches and bloating as well as stomach upset. Of course the data tell a different story: A study published in the early 1990s which attempted to correlate artificial sweeteners (aspartame) with brain tumors (glioblastoma and others) is considered entirely erroneous.

Again it’s the problem of correlation not being the same thing as causality – and when this happens, it means there are unidentified, unmeasured variables in the mix. New York University professor Charles Seife published a paper showing that he could correlate brain tumors to United States government deficit spending. Interestingly, while rates of brain tumors ‘increased’ minimally in the population around the time that certain artificial sweeteners were being used, there was also a parallel increase in the number of MRI imaging machines put into service in hospitals and clinics. So it’s more likely that the increased findings of brain tumors were instead related to… the increased screening for brain tumors.

"Higher" rates of brain tumors over time

“Higher” rates of brain tumors over time

For example, if I plot seven participants, as in the sample size used in one of the diet soda studies, we may see a nice fit to the data pattern, as one I created here:Screen Shot 2014-09-22 at 10.19.16 AM

However, if I include a wider and more representative data set… The pattern disappears!Screen Shot 2014-09-22 at 10.19.35 AM

I’ve put a box around the original, very small, sample size (above) to show you that, in the context of a larger respondent pool, it’s not telling the full story, and the localized model that described the limited data before is now all but useless.

For example: in baseball statistics, if you see a pattern, look at a wider data set to see if that pattern still holds; There are many examples of batting averages of .500, .700, or even higher with very few at-bats – but that pattern never holds as the season wears on.

What does the clinic tell us about real life?

Also, in the world of clinical trials for new drug treatments, both the treatment groups and placebo groups are monitored for side effects – headaches and stomach upset are two of the most-frequently reported symptoms across all clinical trials in the treatment groups and the placebo groups. So people like to report headaches and stomach upset no matter the conditions, and those would be two of the most common psychosomatic effects.

The next big thing beginning to circulate in the nutrition industry is the acronym FODMAP: Fermentable Oligo-Di-Monosaccharides and Polyols. These are short-chain and simple sugars and sugar alcohols (such as sorbitol, xylitol, mannitol, etc. which you can find in, for example, sugar-free chewing gum). The theory goes that these FODMAPs are poorly digested and absorbed in the small intestine, and so some pass to the large intestine where they become fermented and can cause bloating and digestive discomfort. If you look at the list of FODMAP-containing food sources, it’s so broad as to be not realistic as a source of widespread issues. Even non-celiac gluten sensitivity may not exist, as discussed here. In this referenced study, a researcher rotated subjects through a variety of different diets and found “absolutely no specific response to gluten,” which included those subjects who reported ‘gluten sensitivity.’

Don’t claim to find patterns where none exist

There’s an old adage in medicine and healthcare which is appropriate here: When you hear hoofbeats, think ‘horses,’ not ‘zebras.’ There have been so many specific studies performed on artificial sweeteners of all kinds and classes, which had rigorous study designs, and have demonstrated robust safety profiles – like I mentioned above with the thought experiment of the ice water and the unprecedented measurement accuracy our laboratory devices can achieve now, just because we can detect changes at the sub-cellular level doesn’t mean that there is any macro-scale negative impact. The overwhelming factor in weight loss is the simple thermodynamic calculation of calories being consumed versus calories expended (by basal metabolic rate in addition to any activity or exercise).

Diet soda contains no calories so it’s not providing any effect to the left-side of the equation (the calories-in). As far as any changes in the microbiota are concerned: The microbiota changes with any changes to the diet, so again just because we can measure it, doesn’t necessarily mean there’s any overall effect. Similarly, there was a great deal of shock-media from a decade ago about potato chips generating acrylamide during the cooking process which could be carcinogenic – we can try to impute any number of irrelevant risks into as many facets of our life as we choose; But we’d be much better off as a society to deal with our biggest real risks such as by being attentive and wearing seatbelts while driving, exercising for cardiovascular benefits, and vaccinating children.

Ben Locwin, PhD, MBA is a contributor to the Genetic Literacy Project and is an author of a wide variety of scientific articles for books and magazines. He is also a researcher and consultant for a variety of industries including behavioral and psychological, food and nutrition, pharmaceutical, and academic. Follow him at @BenLocwin.

  • Jim Kling

    Where does this article actually address the Nature study? Nowhere as far as I can tell, though the title claims to refute it.

    I have no idea whether artificial sweeteners are bad or not. I avoid them because they taste terrible, which is reason enough for me. The Nature study was done in mice, which has obvious limitations that would be simple enough to address. Instead the author uses obfuscation and is pretty hard to follow.

    • BenK

      The original study does include human data as well as the mouse data. [ 381 non-diabetic individuals (44% males and 56% females; age 43.3 ± 13.2) in an ongoing clinical nutritional study.] It does not use proxies for macro effects – it uses actual macro effects (waist to hip, etc).

      • by jiminy

        Wait, so you’re arguing that the fact that the study documents not just microeffects, but actual increases in obesity and blood sugar, is a weakness in the study? That actual effects are less reliable than proxies? How does that remotely make sense?

        Look, I understand that you are fed up with poorly reasoned, sensational arguments based on shoddy evidence. But that is no excuse to make them yourself. I have no doubt there have been many specious claims about artificial sweeteners in the popular press. But for you dump your frustration about previous stories onto this one, whether they apply or not, is not okay.

        The only part of your critique I could find that specifically addresses THIS paper is when you argued that HbA1C levels are not a reliable indicator of incipient metabolic syndrome — without mentioning that you pulled that out of a long sentence listing many other metrics that also increased, such as obesity and blood glucose, which no one disputes are related to glucose intolerance and metabolic syndrome. Taking this one piece of evidence out of context and implying that their whole argument relies solely on HbAIC levels seems disingenuous if not dishonest.

        The rest of your essay is just straw man arguments. It’s all well and good to remind people that correlation does not equal causation. We all agree with that. But what about the actual causation evidence presented in the paper? You don’t even mention it, let alone critique it; you just hope people won’t read the paper and won’t notice it.

        Likewise starting with the snide comment that it’s not news that something in the diet alters the ratio of one’s microbiota. Who said that was news? Not this paper. Its point is that the changes in gut microbiota resulting from NAS consumption cause — and I use that word intentionally — increases in several metrics for glucose intolerance. I was really hoping for an honest critique of the evidence presented, but all I found were snide remarks and straw man arguments.

        To my eyes, you ended up discrediting not the report, but the reflexive rejection of unwelcome evidence by defenders of the beverage industry.

        I would certainly agree that it’s premature to assign responsibility for the obesity epidemic or the rise in Type 2 diabetes on NAS consumption. But that’s not what the paper did. They just said, here’s some evidence that NAS consumption could be contributing in some way to the very problem they were designed to ameliorate. They’re not saying they cause health problems at the same rate that sugar does, or even that their benefits to society don’t outweigh their costs. They’re just saying, here’s some new evidence suggesting negative effects that need to be studied. Your response–to lump this study together with specious arguments that are more easily discredited–ends up sounding more like an attempt to sweep unwelcome evidence under the rug than an honest appraisal of the facts.

  • Donna Funk

    Don’t know if you read the original article Jim, but he is methodically addressing all of the negatives that were found in this tiny sample of patients and demonstrating that their “findings” are far from scientific. They speculate on the meaning of their data when there is no basis, as this author addresses. People are quite easily confused by correlations – and often attribute differences to causation rather than association. Diet soda does not taste horrible to me. But this kind of research can lead well-meaning people to regulate its use if it is considered valid. Thanks to Ben for showing the problems with this type of research – which is all too prevalent in this day and age.

    • You’re right to point out that this type of research, and the sensationalist media coverage that often accompanies it, is misleading and is not substantiated by the body of science. Fortunately, this article takes a critical look at the methodology and findings of this study, and discusses some of the many credible scientific studies that prove the safety and efficacy of low-calorie sweeteners. Thank you for your comment. – American Beverage Association

    • Jim Kling

      Hi Donna. Well, I was caught having skimmed the original paper and clearly posted too soon.

      I’ve looked at it a bit more carefully and my opinion hasn’t changed much.

      Let’s take a closer look at this blog post:

      Paragraph 1 states his position, that this research is weak. Fair enough.

      Paragraph 2 concedes that one finding of the paper might be accurate.

      Paragraph 3 and 4 cite another study that supports the current one, then goes off on a tangent about cold sensation and cell death in the tongue. This has nothing to do at all with the current work.

      Paragraph 5 and 6 call HbA1c measurements into question. This is an interesting argument that I haven’t heard before. Valid point. It would be helpful if the author had discussed specifically how the original study had used HbA1c in coming to their conclusions, or even defined it. Instead the reader must go to the article to decipher this criticism. If he’s trying to convince lay readers — the ones swayed by all of these internet articles — then this is not an effective way to go about it.

      In Paragraph 7, after 600+ words, we finally get a direct criticism of the study. The author refers to a small sample size of 7 participants. Valid criticism! Only he forgets to mention the 381 non-diabetic participants in a nutrition study. Here’s what the study authors had to say about this group: “We found significant positive correlations between NAS consumption and several metabolic-syndrome-related clinical parameters (Supplementary Table 6), including increased weight and waist-to-hip ratio (measures of central obesity); higher fasting blood glucose, glycosylated haemoglobin (HbA1C%) and glucose tolerance test (GTT, measures of impaired glucose tolerance), and elevated serum alanine aminotransferase (ALT, measure of hepatic damage that is likely to be secondary, in this context, to non-alcoholic fatty liver disease).”

      Could this oversight be related to the information we find at the end of this blog post, which is that Dr. Locwin consults for food and nutrition companies? Maybe, maybe not. But when someone has a conflict of interest, it’s important not to take them too closely at their word.

      In paragraph 8 and 9, under the subhed Symptoms, the author goes off on another tangent, quoting an apparently farcical academic paper published by the eminent science writer Charles Seife. A warning of the dangers of using a small sample size? I guess so. But just adds to the confusion.

      Next we get some graphs. These would make an interesting, short blog post about the risks of small sample sizes. Nice illustration of the problem. Unfortunately it’s buried so deeply in the post and follows so much obfuscation that only die-hard readers are likely to make it this far.

      Next we get baseball. Big sports fan here, but come on…

      And then, God help us, FODMAP. And then horses and zebras.

      So no, not convincing. I’m sure it’s interesting to highly trained professionals who study nutrition, but the target audience for the Genetic Literacy Project is clearly lay people. This post isn’t helping that group of people.

      By the way, if you want good examples of medical study critiques, I recommend I’m hoping they’ll tackle this study, but they haven’t yet.

      • Ben L

        Thanks for taking the time to read Jim. I wouldn’t underestimate the intelligence of the readers of the Genetic Literacy Project, or their ability to draw various layers of knowledge and information from the article. This article is one of the most viewed and re-cited pieces on the site – so the voice of the people is that they want to hear more.

        When teaching physicians and clinicians how to respond to questions about the safety of artificial sweeteners, Pareto analysis shows that more often than not, other dietary questions come up in the clinical visit – including to a large extent now – FODMAPs. Including this logic thread in the article allows readers to appraise similar information in a more robust way.

        As for the sample size of 7, that’s indeed what was measured by researchers Drs. Elinav and Segal – and an effect was seen in 4: not a better result than chance. Additionally, specifically calling into question the measurement of HbA1C is to demonstrate that there isn’t a good deal of data validity and assay reliability in many of the lab results that are taken as ‘gold standards.’ This also ties into the discussion of ‘just because we can measure it, doesn’t mean that it’s relevant’ – not only are many of these assays unreliable, but the ability to detect an effect says not necessarily anything at all about the patient (or respondent); our inter-personal baseline values fall into a range of different levels, and throughout the course of each day, our own physiological measures can vary dramatically. John Ioannidis tackles some of this sector in his papers about test and diagnosis specificity and false positive rates and improper diagnoses.

        The reason why nutritional studies sway from one side to the next on a topic seemingly every 16-18 months is because of all of the other effects in diet and lifestyle that overwhelm the ability to control and measure holistic effects of very small doses of certain nutrients.

        The largest metaanalytic studies of non-nutritive sweeteners – which minimize statistical error much more than a single study – show that they are safe and lead to moderate weight loss – by simple thermodynamics.

        This won’t be the end to the discussion on this topic, but it certainly asks the public and consumers to consider how they appraise information and make decisions in a smart way.

      • Donna Funk

        In the end, no one report has all of the answers. The 7 animal study is hardly worthy of attention but if someone else does a better study to look at this then it may not have been in vain. However there are larger studies indicating that artificial sweetener is safe, and Europe – which has a tendency to be negative first and then look at the research does not have a problem with them. More study needed – but with years and years of safe use, hard to see the problem.

  • As this article rightly points out, there is an extensive body of scientific research, which establishes that diet soft drinks and the low-calorie sweeteners they contain are safe and a viable tool for cutting calories. This new study discussed here has many limitations – from its primary focus on rodents to its use of unrealistic sweetener applications and doses. In sum, this research is not a reflection of real life for humans; nor does it overturn the substantive science that has prompted leading health organizations to condone the use of these sweeteners and the decision of regulatory agencies around the globe to approve them. In other words, it’s overly simplistic and misleading to pin the blame for serious public health challenges on diet soda, which science has repeatedly proven to be safe and an effective weight loss tool: – American Beverage Association

  • UserBob

    As an occasional drinker of diet soda I was interested in this article but come out more confused, especially after reading the comments. So is Dr Locwin misleading us by only citing the 7 instead of the 381?

    Am I to understand that the finding that “artificial sweeteners change the ‘gut micobacteria'” is a mute issue because its always changing based on changing diets? So the anti-artificial sweetener argument is essentially false?

    Some of the science goes beyond me, but when I see the American Beverage Association weighing in -twice- on this, I question the authenticity of the article.

    • Ben L

      To be clear, Drs. Elinav and Segal, in trying to interpret their results in mice across to the physiology of humans, used 7 dosed participants, and noted an effect in 4.

      The ABA seems to have responded because there is a tremendous effect on consumer behavior with poor science, as has been reported about this study in the recent news.

      It’s convenient to discount or not publish in the mass media a much bigger and more statistically-effective study showing that diet sodas indeed have no calories and – surprise – contribute to weight loss:

      “Indeed, the most recent analysis, published in 2014, found that, when considering only the highest-quality evidence, food and drink sweetened with aspartame and the like were associated with a moderate reduction in weight.”

      • by jiminy

        Once again, this is a total straw man argument! No one in their right mind would claim that the obscene amounts in sugar-laden soft drinks don’t cause obesity. Find me the place in this paper where the authors say that non-caloric artificial sweeteners cause as much obesity as caloric soft drinks.

  • Mit Jae

    Like everything else – there is “research” and opinions on both sides of the equation. If you need artificial sweeteners to avoid obesity, you’re drinking too much COKE – I think water might be a god alternative. In general, I’m more comfortable eating and drinking food rather than chemicals despite the many studies that they are just fine. We have evolved with a digestive systems geared towards processing FOOD – but – that’s just my opinion