Evaluation of the FDA Website Usability
Executive Summary
- The FDA declares when they approve drugs but provides encapsulated information around the clinical trials on their website.
Introduction
The FDA is considered the official approving body of drugs in the US, and its influence goes far beyond the US.
- When investigating the FDA website, it became apparent that the FDA publishes remarkably encapsulated information about the clinical trials it reviews.
- Furthermore, the website also has a large number of usability and information findability issues.
I will cover them in this article and provide the website with an overall rating.
Our References for This Article
If you want to see our references for this article and related Brightwork articles, visit this link.
How to Present the Results of Clinical Trial Information
When a drug is reviewed, it means that there are studies that are performed. There are established ways of doing this. Before jumping into the FDA’s website, I wanted to provide an example of one site that does this very well, which is at the site EarlyC19.com.
The example I will use in this article is for treatments for covid. This is just because it is topical. The focus of this article is not the drugs themselves (I address that in a separate article) but how the FDA website enables the information from clinical trials to be accessed.
The C19Early Website
Observe how C19Early presents the different treatments in an easy to review table – which also incorporates a graphic.
This listing of other treatments against Covid 19 shows similar effectiveness, many of which can be taken before contracting Covid. Many of these drugs do not have many studies performed on them.
Here is a comparison table that shows all manner of vital and interesting statistics. I could not find anything like this on the FDA’s website.
Here is another graphic from the website on Ivermectin.
Next is a table.
All of this is easy to follow.
Now let’s see how the FDA publishes information.
Comparing to the FDA’s Website
Now let’s see how the FDA publishes information.
The FDA website has a Covid vaccine page. Then when one selects one of the vaccines, one is taken to that page, which is next. The first vaccine page we reviewed was for Moderna.
The FDA’s website is heavy with technical jargon. And it took a while to read through the material because the material was written as if it is only designed for those with a substantial background in the field. However, this is neither necessary nor what some other website do that cover the same information. See the following text from the C19Early website.
We also perform a simple analysis of the distribution of study effects. If treatment was not effective, the observed effects would be randomly distributed (or more likely to be negative if treatment is harmful). We can compute the probability that the observed percentage of positive results (or higher) could occur due to chance with an ineffective treatment (the probability of >= k heads in n coin tosses, or the one-sided sign test / binomial test). Analysis of publication bias is important and adjustments may be needed if there is a bias toward publishing positive results.
Figure 2 shows stages of possible treatment for COVID-19. Prophylaxis refers to regularly taking medication before becoming sick, in order to prevent or minimize infection. Early Treatment refers to treatment immediately or soon after symptoms appear, while Late Treatment refers to more delayed treatment. – C19 Early
Notice this text is straightforward to read. I will provide more examples of the difficulty level of the text at the FDA’s website.
Overreliance on PDFs
The FDA uses links to several PDFs rather than web pages. It is not at all clear what advantage PDFs provide over standard web pages.
Much of the information is encapsulated within PDF documents instead of web pages.
This PDF document combines several problematic features.
- PDFs consume a lot of space within a browser compared to a webpage. They take a long time to load versus a webpage. Therefore, they are generally inefficient, don’t provide any advantage over standard HTML pages, and have a lower ability to present graphics and no ability to embed videos or other dynamic elements. PDFs have their place, such as for signed legal documents and for glossy marketing type material where the interest is to provide a very polished appearance; however, I cannot think of any advantages for presenting technical information.
- The way the FDA formats is PDFs makes for a poor reading experience. Their PDFs need more line spacing. The entire page is far too crowded with text. This is a 15,000 person organization, and they can’t create a readable PDF document when presenting information is a significant part of what the FDA does.
- The writing within the PDFs is cryptic, and it uses jargon without providing any links to what the jargon is. What are lipid nanoparticles? It would be good if the FDA included a link to a description of what this is. Links can be added to PDFs but work more smoothly within web pages. Many people are interested in this information, not just physicians. Why is this information written in such a cryptic fashion?
- The PDFs go on for many pages like this, and it reduces the interest in completing reading the document.
Here is another example of cryptic writing.
Safety data from approximately 38,000 participants ≥16 years of age randomized 1:1 to vaccine or placebo with a median of 2 months of follow up after the second dose suggest a favorable safety profile, with no specific safety concerns identified that would preclude issuance of an EUA. Available safety data from all participants enrolled through the November 14, 2020, data cut-off (N=43,252, which includes late enrollment of additional adolescent and adult participants), was consistent with the safety profile for the approximately 38,000 participants with median follow-up of 2 months and also did not raise specific safety concerns. The most common solicited adverse reactions were injection site reactions (84.1%), fatigue (62.9%), headache (55.1%), muscle pain (38.3%), chills (31.9%), joint pain (23.6%), fever (14.2%); severe adverse reactions occurred in 0.0% to 4.6% of participants, were more frequent after Dose 2 than after Dose 1, and were generally less frequent in participants ≥55 years of age (≤ 2.8%) as compared to younger participants (≤4.6%). The frequency of serious adverse events was low (<0.5%), without meaningful imbalances between study arms. Among non-serious unsolicited adverse events, there was a numerical imbalance of four cases of Bell’s palsy in the..
This is the type of writing that has to be reread multiple times before it sinks in. Using many numbers in a line of text does not support retention but instead overwhelms the reader.
Keep reading, and it should not take long for your eyes to glaze over.
The EUA request includes safety and efficacy data from an ongoing Phase 3 randomized, double-blinded and placebo-controlled trial of mRNA-1273 in approximately 30,400 participants. The primary efficacy endpoint is the reduction of incidence of COVID-19 among participants without evidence of SARS-CoV-2 infection before the first dose of vaccine in the period after 14 days post-dose 2. In an interim analysis conducted using a data cutoff of November 7, 2020, a total of 27,817 participants randomized 1:1 to vaccine or placebo with a median 7 weeks of ffollow-up post-dose 2 were included in the per-protocol efficacy analysis population of participants without evidence of SARS-CoV-2 infection prior to vaccination. Efficacy in preventing confirmed COVID-19 occurring at least 14 days after the second dose of vaccine as 94.5.0% (95% CI 86.5%, 97.8%) with 5 COVID-19 cases in the vaccine group and 90
COVID-19 cases in the placebo group. Subgroup analyses of the primary efficacy endpoint showed similar efficacy point estimates across age groups, genders, racial and ethnic groups, and participants with medical comorbidities associated with high risk of severe COVID-19. Secondary efficacy analyses suggested benefit of the vaccine in preventing severe COVID-19 (11 protocol-defined severe COVID-19 cases in the placebo group vs. 0 cases in the vaccine group) and in preventing COVID-19 following the first dose, although available data for some of these outcomes did not allow for firm conclusions. Efficacy data from the final scheduled analysis of the primary efficacy endpoint (data cutoff of November 21, 2020, with a median follow-up of >2 months post-dose 2) demonstrated a VE of 94.1% (95% CI 89.3%, 96.8%), with 11 COVID-19 cases in the vaccine group and 185 COVID-19 cases in the placebo group and was consistent with results obtained from the interim analysis. The VE in this analysis when stratified by age group was 95.6% (95% CI: 90.6%, 97.9%) for participants 18 to <65 years of age and 86.4% (95% CI: 61.4%, 95.5%) for participants ≥65 years of age. A final secondary efficacy analysis also supported efficacy against protocol-defined severe COVID-19, with 30 cases in the placebo group vs. 0 cases in the vaccine group, with one severe case in the
vaccine group confirmed after this analysis.
This contains the information, but I predict few people could get through this text. The pharmaceutical company wrote much of this and may have been deliberately written to put people to sleep.
The most common solicited adverse reactions associated with mRNA-1273 were injection site pain (91.6%), fatigue (68.5%), headache (63.0%), muscle pain (59.6%), joint pain (44.8%), and chills (43.4%); severe adverse reactions occurred in 0.2% to 9.7% of participants, were more
frequent after dose 2 than after dose 1, and were generally less frequent in participants ≥65 years of age as compared to younger participants.
What is does between .02% and 9.7% mean because that is quite a range.
It now turns out that there are other side effects not mentioned here.
Finland on Thursday paused the use of Moderna’s (MRNA.O) COVID-19 vaccine for younger males due to reports of a rare cardiovascular side effect, joining Sweden and Denmark in limiting its use.
Mika Salminen, director of the Finnish health institute, said Finland would instead give Pfizer’s vaccine to men born in 1991 and later. Finland offers shots to people aged 12 and over.
“A Nordic study involving Finland, Sweden, Norway and Denmark found that men under the age of 30 who received Moderna Spikevax had a slightly higher risk than others of developing myocarditis,” he said.
Swedish and Danish health officials had announced on Wednesday they would pause the use of the Moderna vaccine for all young adults and children, citing the same unpublished study. – Reuters
The FDA stated that the two-month duration should allow for all safety issues to be caught.
Data from Phase 3 studies should include a median follow-up duration of at least 2 months after completion of the full vaccination regimen to help provide adequate information to assess a vaccine’s benefit-risk profile. From a safety perspective, a 2-month median follow-up following completion of the full vaccination regimen will allow identification of potential adverse events that were not apparent in the immediate postvaccination period. The study was sufficient to observe the side effects.
Well, not. It was insufficient.
But why has this document not been updated after this new information about other side effects came to light?
An approval document, called the EUA, does not have to be a dead document after it is written. It can be continually added to and can have links out to the safety issues or other changes. If a lawsuit is filed against the drug manufacturer for a drug, that should be included in the EUA. If a drug’s approval is retracted, that should also be linked to the original EUA. These are all reasons the EUA, and in fact, very few of the FDA’s documents should be in PDF documents. PDF documents do not naturally lend themselves to further additions. Everything the FDA publishes should be subject to improvement and new information as it comes to the FDA.
The FDA shows the three studies submitted to them by Moderna.
This is essential information. How many trials were submitted to the FDA? Three trials were. However, one of them only had 120 people in it.
I had never heard how Moderna submitted many trials before their vaccine was approved. Why is the only place this is explained or listed is within the EUA?
This brings up the question of how many people, even in the media, are reading these approvals that the FDA is publishing.
Finding the Pfizer EUA?
I spent 40 minutes going through the FDA’s on-site search, and I could not find the EUA for the Pfizer vaccine. This is the approving document that contains information about the clinical trial. Why did it not come up? The one that I did find was for a different approval, not the original approval.
Also, I searched Google and found pages that announced the approval, but none of these pages linked back to the original Pfizer EUA for the vaccine. This is one of the most important documents on the FDA website for now (the EUA for the Pfizer, AstraZeneca, and Moderna vaccines being the three). Why is it so difficult to find?
Finally, I found it by rereading the links on the original Pfizer Covid vaccine page.
This is the Pfizer-BioNTech page at the FDA. Where is the EUA that is what approved the vaccine? Is this not of interest to the FDA, which millions are taking because of this FDA approval not of interest to the FDA to make it easy to find?
All of these links point to documents about this vaccine but not the original approval.
I thought I would contact the FDA and ask them. However, I found that…
- There is no chatbox on the website.
- The FDA has close to 15,000 employees but has no contact box on their website and no staff.
- They also had no one I could speak to, as I was put through to voicemail. I never received a callback.
But I was finally able to find the EUA, and its title link was FDA Decision Memorandum. That is not appropriately named. It needs to have EUA in the title.
Reading the Pfizer EUA
This table shows there were two studies. Again, I never heard how many studies were submitted to the FDA. This is a critical piece of information, but I could much more than those that specialize in this area for their jobs have any idea of this.
This table from the Pfizer EUA PDF shows more of the study details. 
And this study shows the number of study subjects, which was around 40,000, with each study having a little less than 20,000.
Statistics on the Vaccine Clinical Trials
This means that the two most widely used vaccines (Moderna and Pfizer) have five total studies to support their use, 3 (although two as one of the studies only had 120 participants) for Moderna’s vaccine and 3 for Pfizer’s vaccine. The Moderna vaccine had more than 30,000 test subjects, while Pfizer had a little less than 40,000.
Why is this information not aggregated somewhere at the FDA website rather than requiring me to do this math? Let us review again how tables are used effectively at the C19Early website.
How about an analytical table like this aggregating the studies for the two most widely taken vaccines in the US? This is a problem with the FDA website. The information is kept within PDF documents, but there are no analytics or graphics that aggregate information from the studies published outside of the PDF documents.
Where is the Explanation of the Financial Bias in the Studies
Most of the studies presented to the FDA are presented by pharmaceutical companies that seek approval to make money on drugs. Part of research integrity is to describe one’s financial interests in the study. However, the FDA never discusses or even mentions the financial biases that the pharmaceutical companies have that submit studies to them.
Why is this? Is the FDA unaware that a foundational rule of research integrity is to publish one’s financial conflicts? Are these studies not 100% funded by profit-seeking pharmaceutical companies? Yes? Then there is no reason, outside of corruption, for the FDA to not make this clear to readers.
The drug companies have a strong financial incentive to get their drugs approved as it means enormous sums of money for the drug company. However, the FDA’s website does not mention this conflict of interest anywhere. The FDA might also want to mention its own financial bias in approving drugs as 45% of its budget is funded by pharmaceutical companies (this was not always the case, but changed in the 1980s under pressure from AIDS activists to get AIDS treatments approved faster). This means the FDA should be transparent that it is only partially a government agency.
Reviewing the FDA’s Website Purpose
The FDA’s website seems to have less to do with communicating analytical information than promoting the FDA. One article after another is around how the “FDA is taking action” by approving this or that drug. It is a problem when “taking action” is synonymous with endorsing a drug. Taking action can also mean not approving a drug.
The FDA website is anti-analytical. The FDA does not know how to present information effectively or encapsulate its information to dissuade people from reading its publications.
Conclusion
When compared against the C19Early.com website, which tracks many treatments for covid 19, the FDA website is inferior and is far less analytical than the FDA’s website. The FDA presents a lot of information in their EUAs and other documents, but it is not analytical. They appear more adept in just reporting raw data than in providing analysis.
The FDA has a budget of $6.5 billion. However, they have a website of what looks like a much smaller and less substantial entity. The FDA does not take seriously educating the public and making the information on its site easily accessible, which drives down all the way even to how the FDA writes their documents. The FDA’s website is virtually the only way that the FDA can communicate information of any detail to the public and representatives from the FDA are rarely if ever seen on public forums or in interviews with the media.
We rate the FDA website a 3 out of 10 for usability and findability of information on the site.
Visit Our Subscription Website on Ivermectin and Other Medical Treatments
Ivermectin has many treatment applications outside of its approved use (as an antiparasitical. These treatments include cancer, covid, immunity, and more. And due to perverse financial incentives, many of these applications are suppressed.
We have created a subscription website that covers everything related to Ivermectin ranging from its many health improvement applications to dosages, and contrasting this with the inaccurate information presented on Ivermectin by medical authorities. And also other topics such as immunity and cancer. The site focuses on overall health improvement and specific treatment analysis.