A new study on sugar can best be said to illustrate the decline in our educational system, which is failing to teach the scientific process and increasingly leaving students and professors unable to design, conduct, evaluate and report a clinical study that is a fair test of an hypothesis. That may sound like an exaggeration until you read the study. It also serves as a reminder of the importance for all of us to keep our critical thinking caps on, and not trust professional peer review or media to think for us.
Most of us probably first learned about this research from a news headline: “A new study has found that we can boost our reaction times, improve our focus and self control by gargling with sugar water.” Like virtually all stories in the news today, the source of this story was a press release, Sugar boosts self-control, UGA study says, which most media reported verbatim.
To boost self-control, gargle sugar water. According to a study co-authored by University of Georgia professor of psychology Leonard Martin published Oct. 22 in Psychological Science, a mouth rinse with glucose improves self-control.
His study looked at 51 students who performed two tasks to test self-control. The first task, which has shown to deplete self-control, was the meticulous crossing out of Es on a page from a statistics book. Then, participants performed what is known as the Stroop task where they were asked to identify the color of various words flashed on a screen, which spell out the names of other colors. The Stroop task’s goal is to turn off the student’s tendency to read the words and instead see the colors. Half of the students rinsed their mouths with lemonade sweetened with sugar while performing the Stroop test, the other half with Splenda-sweetened lemonade. Students who rinsed with sugar, rather than artificial sweetener, were significantly faster at responding to the color rather than the word….
They theorized that the glucose causes emotive enhancement, leading the person to pay attention to their goals and perform better at evoking the non-dominant response.
News media, like the New York Daily News, was quick to play up the study and focus on the authors’ speculations on the reason for their findings.
“Researchers used to think you had to drink the glucose and get it into your body to give you the energy to [have] self control,” says coauthor Leonard Martin, professor of psychology. “After this trial, it seems that glucose stimulates the simple carbohydrate sensors on the tongue.” He adds: “This, in turn, signals the motivational centers of the brain where our self-related goals are represented.” And it’s these signals, he adds, that wake your body up and tell it to start paying attention.
But we didn’t get the full story from the news, or even enough information to evaluate it. Only by going to the study, “The Gargle Effect: Rinsing the Mouth With Glucose Enhances Self-Control,” is it possible to gauge the study’s credibility.
In this study, the authors — University of Georgia psychology professor, Leonard Martin and doctoral candidate, Matthew Sanders — hypothesized that glucose can enhance self control even when it isn’t ingested and raise blood sugars. They attempted a conceptual replication of an earlier study by Northwestern University psychologist, Daniel Molden, which first proposed an alternative motivational model for the effects of carbohydrates they observed on self-control using the Stroop task.
The authors signed up 51 college students who were required to participate in order to fulfill a course requirement. “We placed no restrictions on who could participate or when,” stated the authors. They omitted requirements such as excluding severely underweight students and those who’d eaten recently, nor did they worry about not conducting the tests extremely early or late in the day.
They had the students rinse their mouths with either sugar-sweetened lemonade or Splenda-sweetened lemonade, while immediately performing the Stroop task. In this task (as shown here), participants are asked to identify the colors of the words that appear on a computer screen as fast as they can. The words they see on their screen may or may not match the meaning of the word (such as the word “green” appear in green type). Reading proficiency among people becomes an automatic process with practice. The Stroop effect interrupts the automaticity of these cognitive processes. In this study, the students responded by pressing labeled keys on a keyboard. Then, the participants rated the sweetness and enjoyment of the lemonade.
There was an 85 millisecond difference in reaction times between the students gargling with sugar versus Splenda sweetened lemonade. The authors concluded that since it takes glucose 10 to 15 minutes to enter the brain after being ingested, that the glucose affected self control not by affecting metabolism, but by activating the brain.
But is that what their study actually demonstrated? Was this a fair test of the hypothesis? The scientific process has proven for centuries to lead us to the soundest conclusions. By failing to follow the basic elements of the scientific process, poor quality or weak studies are not “fair tests” of an hypothesis and cannot be relied upon. It’s easy to conduct a trial, intentionally or unintentionally, that is biased and more likely to support an hypothesis.
Most of you have probably already identified many of the methodological flaws and biases in this published trial. Nowhere, was there any mention of it being a randomized, placebo-controlled, double blind trial. These design failures inserted numerous experimental biases that invalidated the ability to make any sound conclusions. Let’s look at a few.
This study used nonvolunteer students without restrictions and introduced a host of selection or allocation biases to the study.
- Experimental evidence has shown that older adults show more of a Stroop effect than younger participants and that all people improve on the task with practice.
- Use of nicotine, amphetamines, cannabis, coffee, alcohol, sleep patterns, nutrition or dieting, fatigue, recent participation in sports, general intelligence and reading skills, language barriers, medical treatments, physical or mental illness, anxiety and depression are all known confounders that can affect the Stroop task and cognition, attention levels, motivation and self control.
But, the authors made no attempt to identify these validity problems or control for them in their selection of study participants or interpretation of their findings. The authors also failed to measure or control for the attitudes of the students, such as those who reluctantly participated because it was a class requirement versus those eager to volunteer.
There are countless ways that two groups of people can differ — known, unknown or unmeasureable confounders— that can affect a study. The only way to help ensure that two groups are similar in order to help isolate an intervention effect is to randomly select people to each group. A nonrandomized trials hopelessly suffers from selection bias.
Study results are also biased when the researchers or participants know what is being studied or which participants are in the test versus control group. By failing to double blind this trial, it risked being flawed by both investigator (or observer) bias and expectation (or response) bias.
Study authors are more likely to interpret their observations as being due to the intervention when they know which participants received the intervention. In this trial, the authors noted that the students clearly knew if they were receiving the sugar sweetened lemonade or the Splenda lemonade mix. Students may also, intentionally or unintentionally, have responded in the way they were expected to or have wanted to appear cooperative and help the grad student get the results he was looking for.
It’s also unknown what these participants were told about the study and its hypothesis, increasing the potential for expectation bias. For the past several decades, increasing research has challenged the robustness of the Stroop effect and have been finding that a number of factors can affect automatic cognitive processes, including attention level, memory and mood. Even the power of suggestion can override the Stroop effect. These known confounders accentuate the flaws brought to this trial by not being double-blinded.
The Stroop interference effect has also been shown to increase with stimuli, especially with negative stimuli, that even briefly distract our attention. This study design failed to identify or control for other potential factors that might have interfered with the students’ concentration and response times, such as the room’s temperature, lighting, computer keyboard, noise level, interruptions, etc. Most significant, was the use of Splenda lemonade mix as their control. Have you tasted Splenda lemonade mix? Even if most of participants found the taste acceptable by the end of the study, for a brief moment when beginning to gargle with it while starting their tasks, their faces were probably still puckered up. How might their reaction times have compared against those gargling with plain water, for example?
So, there was an 85 millisecond difference in the response times between the group gargling with sugar sweetened lemonade versus Splenda lemonade. The authors set out to look at glucose and, as interpretor bias does, they interpreted their findings as supporting their hypothesis. Instead of interpreting the findings as showing that sugar improved reactions, however, their results could have just as easily been interpreted as showing that Splenda lemonade mix interfered and slowed reactions.
The authors concluded that their results “provide further support for the hypothesis that glucose moderates self-control nonmetabolically.” But by ignoring the scientific process, this study provided no support at all.