USDA Gets a Failing “GRADE” on Science
By Emma Hitt Nichols, Ph.D.
In oral testimony recently to the Dietary Guidelines Advisory Committee (DGAC) in Washington, D.C., Nina Teicholz, Executive Director of The Nutrition Coalition, emphasized that the Guidelines have historically excluded or ignored the vast majority of clinical-trial research on nutrition and also—citing expert opinion—that the scientific methodology of the Guidelines fails to meet basic scientific standards.[1]
Teicholz, in her 3 allotted minutes, made two important points. First, she said, the Dietary Guidelines have, since their launch in 1980, ignored or excluded nearly all the rigorous clinical-trial literature on fat, saturated fat, and health. Teicholz’s second point was that the USDA officials directing this process have decided to use a “modified” version of GRADE, a rigorous standard for rating the quality of evidence that is not meant to be modified.
Teicholz reported that the co-founder of GRADE himself, Dr. Gordon Guyatt, had recently submitted a public comment stating that USDA’s modifications to GRADE were so significant that the office should not use the term “GRADE” because doing so would create an “illusory” appearance of scientific rigor where, in fact, none existed.
The advisory committee in charge of reviewing the science for the next Dietary Guidelines for Americans met at the United States Department of Agriculture (USDA), July 10-11, 2019. This was the second of five committee meetings in the process to review the science for the next set of Dietary Guidelines (DGA), due out in 2020.
While all of these meetings are open to the public, this was one of only two meetings where the public is invited to make oral comments to the Committee.[2]
Important Data Excluded
Teicholz spoke to the fact that the Guidelines, for the past 35 years, have ignored clinical trial evidence, mostly funded by governments around the world, on more than 75,000 people, in studies lasting up to 12 years. These numbers come from a peer-reviewed 2015 article in The BMJ, authored by Teicholz.[3] See box for updated information on these excluded studies. The exclusion of these data is an especially serious omission, because this kind of evidence, from clinical trials, is considered the “gold standard,” since it can uniquely demonstrate causality.
Update on studies excluded from all DGA scientific reviews
- Total volume of research excluded [from 85 studies]: 906 total pages of excluded or ignored research.
- Data from clinical trials (the "gold standard") excluded: 136,780 people studied over a total of 139 years.
- Cost considerations: the cost of all the excluded studies is not available, but for a fraction of the studies (17), the cost of this research was $904,234,637, suggesting that the overall cost of excluded studies is likely to be in the billions of dollars.
“Unfortunately, instead of informing our nation’s nutrition policy, this gold standard evidence has been ignored,” Teicholz said in her oral comments.
“Why? One can only speculate,” said Teicholz. “Yet the fact is that none of these trials could confirm the basic tenets of the Dietary Guidelines. Indeed, multiple trials did not confirm that a diet restricted in fat or saturated fat could protect against diet-related diseases. More recently, trials have shown that the Guidelines’ high level of carbohydrates [51-54% of daily calories] actually is harmful for people with diet-related diseases.”
According to Teicholz, if this trial evidence had not been excluded, the Guidelines would have “walked back” some of their recommendations. “Yet the guideline experts ignored the evidence and carried on without change,” she said.
“Diet-related diseases now affect at least 60% of all Americans. Ignoring scientific evidence is not okay,” she said.
NASEM Report Supports Nutrition Coalition’s Concerns
Teicholz’s concerns are echoed by the National Academies of Sciences, Engineering and Medicine (NASEM), which, in a 2017 report, provided the first ever outside peer review of the DGA.[4]
The Academies concluded that the process for the guidelines was not using the best practices for conducting systematic reviews and “lacked scientific rigor.”
The Academies advised that the USDA adopt one of the international standards for systematic reviews of the science. At the first DGAC meeting, in March, USDA staff announced that it intended to use a “modified” version of GRADE, an internationally recognized methodological standard for reviewing scientific literature.
GRADE Co-Founder Says USDA’s “Modified GRADE” Is Not Acceptable
However, as Teicholz noted in her comment, the co-founder of GRADE himself, Dr. Gordon Guyatt, in a recent written public comment to the USDA, expressed strong concerns about the USDA’s use of a “modified” version of GRADE.[5]
Guyatt cited, as a principal problem, USDA’s lack of a methodology to distinguish between high- and low-quality evidence: “This distinction between high- and low-quality evidence lies at the core of any rigorous evaluation of science and is at the heart of the GRADE methodology,” wrote Guyatt. A Distinguished Professor in the Departments of Health Research Methods at McMasters University, Guyatt urged the USDA not even to use the word GRADE “because doing so would give the appearance of rigor where it did not exist.”[5]
Further, Guyatt cautioned that if the USDA were to continue without proper methods for evaluating the evidence, the result would be recommendations that are, “unlikely to be trustworthy.”
Teicholz, in her oral comment, warned about the danger of a future Dietary Guidelines based once again on weak evidence that would not be trustworthy—and thus, not serve the public health.
“The USDA has been admonished by the National Academies of Sciences and encouraged to improve. Unfortunately the Guidelines are on track to repeat the same mistakes of the past.” she concluded.
At least a third of the commenters at the meeting were from industry. Another sizeable group were from non-profits that do not state their sources of funding. By contrast, Teicholz and The Nutrition Coalition receive no industry support. The Coalition is committed to ensuring that America's Dietary Guidelines are based on solid, rigorous evidence, purely in the interest of good science and the public health.
References:
2020 Dietary Guidelines Advisory Committee Second Meeting. Nina Teicholz Commenter 28. http://bit.ly/DGA2ndMtgTeicholz [1:25:21 to 1:28:28]. Accessed July 15, 2019.
https://www.dietaryguidelines.gov/most-popular-questions#DGprocess
Teicholz N. The scientific report guiding the US dietary guidelines: is it scientific? BMJ. 2015 Sep 23;351:h4962. doi: 10.1136/bmj.h4962. https://www.bmj.com/content/bmj/351/bmj.h4962.full.pdf
Comment on FR Doc # 2019-12806. Dr. Gordon Guyatt. https://www.regulations.gov/document?D=FNS-2019-0001-6984
About the Author:
Emma Hitt Nichols, Ph.D., is a health writer and founder of BestShape50.com.