Two Years Too Late? Researchers Announce Hoped-For Results, Stall on Revealing Actual Data
Taken from the November/December 2014 issue of the Women's Health Activist Newsletter.
Unfortunately, most women don’t know that HT’s benefits were over-inflated. And, when it comes to healthy women taking powerful drugs to prevent illness, what you don’t know can hurt you.
In this article, we examine how women were let down by researchers who announced study results to the press, and then failed to back up their claims by providing solid data about the research. We’ve also created consumer health information entitled, “Menopause Hormone Therapy: Timing Doesn’t Matter,” to explain the study design and what the evidence actually shows; it’s available on our website, https://nwhn.org/MHT-Timing-Doesnt-Matter. (Also be sure to read the article in this issue on non-hormonal treatments for menopause symptoms!)
What went wrong? Why did researchers announce “results” that weren’t backed up by data? And, why did their announcement get so much attention from the media?
The Kronos Early Estrogen Prevention Study (KEEPS) was an important study. It was designed to explore whether menopause HT, when given to women shortly after menopause, could prevent cognitive decline and lower the risk of heart disease. The study design involved giving women either menopause HT or a placebo, and then their cardiovascular health and cognition were analyzed at various times throughout the study. KEEPS was funded by a private foundation, the Kronos Longevity Research Institute, without drug company money. The National Institutes of Health (NIH) funded the study of cognitive effects.
KEEPS was started in 2005, a few years after the Women’s Health Initiative (WHI) results made world-wide news with its findings that menopause HT created more health problems than it prevented in healthy women, including cardiovascular problems, such as blood clots and strokes. Proponents of menopause HT argued that women in the WHI started taking menopause HT too long after menopause, and it was the timing, not the treatment, that harmed women. KEEPS was designed to test the effect of starting HT shortly after menopause. Researchers made an effort to get publicity for KEEPS, in part, to encourage women to volunteer for it. The final results of the KEEPS study would have gotten attention no matter where or when they were released. Ironically, though, for a major research study all about treatmenttiming, the researchers manipulated the timing of the results in a way that allowed them to exaggerate and mislead women about HT’s benefits.
The KEEPS researchers didn’t wait until the study had been published in a peer-reviewed journal, as is usually the practice, to start talking about the results. They presented their version of the results at the October 2012 annual meeting of the North American Menopause Society. Their presentations were given on the conference’s opening day, at the Presidential Symposium,1 sending a strong signal to attendees that this was important new information. But, the difference between a conference presentation and publication in a scientific journal is huge. Before an article is published in a reputable medical journal, it is goes through peer review, a process that is intended to ensure the research is valid, and all findings are supported by the study data.
In addition to their conference presentation, which was open to the press, the KEEPS researchers actively sought media coverage by distributing a press release and holding conference calls with reporters.
The most egregious claims were made in KEEPS’ press releases — and the media picked them up. One 2012 press release said that KEEPS found “improvements in cognition” in women assigned to menopause HT, and “slight evidence that HT might be cardio-protective in this age group.”2 That message was widely broadcast on TV, daily newspapers, and on-line medical news outlets, including Medscape and Medpage. “Benefits seen in hormone use early in menopause” ran the headline in USA Today3 and on TV;4 “HRT Benefits Newly Menopausal Women” was the way Medscape described its story about KEEPS.5
But what did the data actually show? Even back in 2012, the NWHN staff figured out that the data told a very different story than the headlines. Here’s the headline that should have been written: “Study finds hormones don’t protect heart in newly menopausal women.”
How could NWHN staff figure out what health journalists didn’t? We’d been following KEEPS since it was first announced and were very interested in the study’s main question (whether hormones given to women shortly after menopause could protect the cardiovascular system). We knew that for researchers to say that the HT treatment used in the study worked, they needed two things. First, the difference in outcomes between the treatment and the placebo groups had to be so strong that it could not have happened by chance. Second, the proof that HT worked needed to be seen where the study was designed to look for it, what researchers call the “primary endpoint.”
The KEEPS study was designed to show whether menopause HT prevented the buildup of plaque in the blood vessels near the heart, thereby promoting women’s cardiovascular health.6 It didn’t. When we carefully read the materials released by the researchers, we saw that the data showed there was no difference in ultrasound measurements of the thickness of the blood vessels between the HT and placebo groups. There were, however, a few differences in other measurements that were not the primary endpoint, and it was these differences that the researchers referred to when they said KEEPS found “slight evidence that HT might be cardio-protective in this age group.”7
The media coverage might not have been as glowing back in 2012 if all the researchers talked about were the supposed cardiovascular results, but they also talked about HT’s benefits on cognition and mood.
The second question that the study was designed to answer was whether this type of hormone treatment protects women against cognitive decline. This ancillary study was funded by the NIH’s National Institute of Aging. Study participants took a comprehensive battery of tests at the beginning, middle, and end of KEEPS to determine whether menopause HT changed their mood and/or improved any aspect of their memory, reasoning skills, or ability to focus.8 In all, the researchers used nearly 20 different tests to look for possible effects of the treatment.
Guess what happens when you give 700 women 20 tests 3 times each? You find differences! No matter what treatment is being studied, if a study includes enough measurements, the treatment will appear to be different at least some of the time.
So, back in 2012, when the KEEPS researchers announced that: “The data showed improvements in cognition…” well, yes, there probably were some data that showed improvements in cognition. And, there probably were some data that showed HT had no effect. And, some data that showed that menopause HT made things worse! Statistics professors have a name for this kind of selective reporting. They call it cherry-picking the data. We call it an outrage!
Researchers had to know that they were selectively reporting positive results from a huge data set. We were suspicious at the time, but wanted to wait until KEEPS’ results were published in a peer-reviewed journal before we weighed in with our analysis.
But now, two years later, where are the results? The results of the heart disease prevention study have finally been published — and they confirm what the Network thought back in 2012. Researchers were grasping at straws when their press release announced that there is “slight evidence that HT might be cardio-protective in this age group.” What they said in the published article that was reviewed by their peers is quite different: “Conclusion: Four years of early [Menopause] HT did not affect progression of atherosclerosis, despite improving some markers of CVD risk.”9
When it comes to the data about cognitive effects — where we think the researchers most strongly exaggerated their 2012 claims and where a peer-reviewed publication is most badly needed — we’re still waiting. That’s right. Two years later, still no publication about the cognitive data. We can only speculate, but perhaps the KEEPS researchers are finding it a bit difficult to present their data and findings in a fashion that’s acceptable to a reputable scientific journal.
We’ll keep waiting, but in the meantime, we want to set the record straight. It is scientifically and ethically wrong for researchers to make exaggerated claims about the supposed benefits of HT in press releases and conference presentations. Women deserve better. And two years is too long to wait for the truth.
Cindy Pearson was the NWHN’s Executive Director from 1996 to 2021. One of the nation's leading advocates for women's health, Cindy often testified before Congress, NIH and the FDA and was frequently featured in the news as a consumer expert on women’s health issues. When she retired, Cindy received a Congressional Resolution in honor of her outstanding contributions to the health of women and girls.
The continued availability of external resources is outside of the NWHN’s control. If the link you are looking for is broken, contact us at nwhn@nwhn.org to request more current citation information.
References
1. Kronos Longevity Research Institute, “Plenary Symposium #1—Presidential Symposium: New Findings from the Kronos Early Prevention Study (KEEPS) Randomized Trial,” North American Menopause Society 23rd Annual Conference, October 6-12, 2012. Available at http://www.menopause.org/annual-meetings/2012-meeting/meeting-abstracts. Accessed 9/20/14.
2. Kronos Longevity Research Institute, “Press Release: Hormone Therapy Has Many Favorable Effects in Newly Menopausal Women: Initial Findings of the Kronos Early Estrogen Prevention Study (KEEPS),” Tempe AZ: Kronos Longevity Research Institute, October 3, 2012.
3. Painter K, “Benefits seen in hormone use early in menopause,” USA Today, October 4, 2012. Available online at:
http://www.usatoday.com/story/news/nation/2012/10/03/menopause-hormone-t.... Accessed 9/20/14.
4. Associated Press, “Benefits seen in hormone use early in menopause,” Fox News, October 3, 2012. Available online at:
http://www.foxnews.com/health/2012/10/03/benefits-seen-in-hormone-use-ea.... Accessed 9/20/14
5. Lowry F, "HRT Benefits Newly Menopausal Women, No CV Harm," Medscape Medical News, October 3, 2012; available online at:
http://www.medscape.com/viewarticle/772061. Accessed 9/20/14
6. Miller VM, Black DM, Brinton EA, et al. “Using Basic Science to Design a Clinical Trial: Baseline Characteristics of Women Enrolled in the Kronos Early Estrogen Prevention Study (KEEPS)” J of Cardiovasc. Trans. Res 2009; 2:228-239.
7. Kronos Longevity Research Institute, “Press Release: Hormone Therapy Has Many Favorable Effects in Newly Menopausal Women: Initial Findings of the Kronos Early Estrogen Prevention Study (KEEPS),” Tempe AZ: Kronos Longevity Research Institute, October 3, 2012.
8. Wharton W, Gleason CE, Miller VM, et al. “Rationale and Design of the Kronos Early Estrogen Prevention Study (KEEPS) and the KEEPS Cognitive and Affective Sub Study (KEEPS Cog).” Brain Res 2013; 1514:12-17.
9. Harmon SM, Black DM, Naftolin F, et al. “Arterial Imaging Outcomes and Cardiovascular Risk Factors in Recently Menopausal Women,” Ann Intern Med. 2014;161(4):249-260. doi:10.7326/M14-0353.