Undergrads Hone Scientific Citizenship Analyzing COVID Trial

April 30, 2021

The COVID-19 pandemic has revealed substantial public mistrust and misunderstanding of iterative scientific research, underscoring the importance of cultivating scientific literacy. We designed a remote, in-depth discussion activity to guide first-year biology students through an analysis of a recent clinical trial publication on remdesivir as a COVID-19 treatment, published in the New England Journal of Medicine (NEJM). The activity prompted unexpected, yet fortuitous, discussions on larger topics, including how science is communicated, how study parameters (including disparities in clinical trial subject demographics) influence results and how allocation of limited resources affects scientific progress.

Motivation and Design of COVID-19 Clinical Trial Class Discussion Activity

Many biology undergraduates seem eager to approach COVID-19-related news with freshly-trained scientific eyes, but are not sufficiently prepared by prior coursework to do so. At the University of North Carolina Asheville (UNCA), the first-year biology course Experimental Design, Analysis, and Presentation is an ideal venue for introducing primary literature that has prompted “viral” news articles in the popular press. The course addresses qualities of rigorous science: alignment of experiments and study objectives, controls, randomization, minimization of bias, sufficient replication and appropriate methods of analyses. Large-scale clinical trials, with human subjects, must take extra care with study design for statistical and ethical reasons, and offer the opportunity for students to probe larger topics connecting science with society.  

The remdesivir COVID-19 clinical trial chosen for the discussion activity embodied the concepts emphasized in class. The activity was implemented virtually and synchronously, using the student response system (SRS) Pear Deck and video conferencing software. An advantage of Pear Deck, as a Google Slides add-on, is the ease of embedding a variety of question types in presentations, including questions where students can sketch directly on slides (beneficial for figure analysis). SRSs promote engagement, original thought, accessibility and inclusion because students prepare and share their responses before seeing those from classmates, shared responses are anonymized and all students can contribute equally to the discussion.

In addition, the anonymity associated with Pear Deck responses and a virtual classroom facilitated an open and honest exchange of ideas. While the original activity goal was to evaluate study design and interpret statistical and graphical data, the open-ended nature of many prompts led to unexpected comments and questions on the study’s broader societal implications, and about biomedical research in general. 

The Activity Promoted Engaged Discussion of Scientific Communication and Public Policy

The past year has provided a rare opportunity to see the process of biomedical research play out rapidly, in real time and under unprecedented public scrutiny. Although the NEJM article chosen for discussion now includes a final report on the remdesivir clinical trial, this activity was initially conducted when only the preliminary report was available. The sequential reporting of the study’s preliminary and final results allowed instructors to emphasize the importance of reevaluating published records as mature data become available. Students insightfully observed that buzzworthy, yet premature, study interpretations sometimes draw louder media attention than more reliable final updates or corrections.

The preliminary report, in particular, promoted discussion of important ethical and scientific questions. For example, how do medical and public health professionals use preliminary data to provide care and inform public policy? Preliminary data, by definition, provide an incomplete picture of a drug’s effects, yet these data were published because the promising results were deemed valuable to share at that time by the authors and NEJM editors. Here, the trial was unblinded early when preliminary analyses revealed that remdesivir significantly benefited patients. In other cases, early analyses may detect emerging negative outcomes or significant side effects. Students discussed the importance of an independent data and safety monitoring group in making these decisions and raised questions about whether preliminary release of results is typical for all clinical trials or related to the urgency of the pandemic. News articles touted the design of this remdesivir study as an adaptive clinical trial and future class discussions could evaluate the scientific and societal merits versus tradeoffs of adaptive clinical trial design—with built-in considerations for expanding sample sizes, ending early or modifying treatment arms based on pilot results—relative to more rigid traditional clinical trial designs.

Pear Deck prompt (top) with student responses (bottom).
Pear Deck prompt on importance of data and safety monitoring groups (top) with student responses (bottom).
Source: Melinda Grosser and Rebecca Hale.

A related discussion highlighted that the rapid pace of new preliminary data arriving on the scene is generally positive, but also has (unsurprisingly) resulted in contradictory studies that receive considerable and often misleading media coverage. One example was a conflicting study cited in the NEJM article’s discussion, which did not find remdesivir to significantly benefit patients. In comparing these studies, students were prompted to consider the impact of study design on results and statistical power (e.g., the other study was not fully enrolled and was designed with unequal allocation into experimental and placebo groups). Satisfyingly, students then took the discussion beyond the initial prompt. For example, how should one interpret data from studies that have inherent (but in some cases, unavoidable) design problems, especially when that may be the only data available? One student specifically referred to data available at the time on another potential COVID-19 drug, ivermectin; its widespread use in Latin America has been cited as an obstacle to performing controlled, large-scale clinical trials on ivermectin of a similar scale to the NEJM article’s remdesivir trial because of limited ability to recruit patients who had not already taken it. An additional class activity that focuses on comparing conflicting studies, or examination of a study with major design flaws, would make for a nice extension activity.

Students brought up further questions about how public resources are allocated to develop treatments. With conflicting preliminary data, how do researchers decide what are the most promising leads to follow? Students were prompted to consider the justifications for the remdesivir study cited in the NEJM article. Another discussion could explore the substantial attention hydroxychloroquine received due to preliminary, small-scale studies of its effectiveness as a treatment. For example, one analysis of all unique COVID-19 clinical trials in April 2020 found that ~12% mentioned chloroquine or hydroxychloroquine. Was such substantial investment of resources into a single potential treatment based on the quality of scientific evidence or public and media hype? Did this focus limit exploration of other, potentially more beneficial, treatment options?

Student responses to a prompt on the rationale behind the clinical trial.
Student responses to a prompt on the rationale behind the clinical trial.
Source: Melinda Grosser and Rebecca Hale.

Discussing Equity and the Use of Racial Categories in Biomedical Research

Another significant topic raised was, what underlies outcome disparities across study groups, and what are their implications? Both the preliminary and final NEJM article reports revealed significant treatment benefits overall and among white patients, but not among subgroups of patients identifying as Black, Asian, Hispanic or Latino, which had smaller sample sizes. We highlighted racial and geographic recruitment considerations and outcome disparities in the context of response rate ratios, confidence intervals and sample sizes. However, this subject piqued student interest and prompted follow-up questions that invoked the growing realization in the scientific community that use of racial groups to approximate genetic similarity in biomedical research is not supported by population genetics data. Some students had read that specific racial and ethnic groups were disproportionately affected by COVID-19—was this due to biological or other (social, societal, environmental) reasons? Are there examples of other diseases or disease treatments that differentially affect particular racial or ethnic groups? Students were interested to learn that while genetic risk factors are emerging for COVID-19, differences in outcome associated with race more likely relate to institutionalized racism, social structure and healthcare disparities rather than genetic variation.

This line of spontaneous discussion was unfortunately constrained by time, but student engagement and the importance and depth of this issue merit revisiting it. Recent articles and opinion pieces highlight troublesome recruitment disparities in COVID-19 clinical trials and biomedical research generally. Future guided discussions should also tackle the problematic use of racial designations as a means to categorize biological diversity in clinical and biological research, and how this might be addressed going forward.

Continuing Discussions

Feedback indicated that students both enjoyed discussing the article and felt that they improved their ability to evaluate study design and interpret data. Considered together, their questions suggest a benefit for including iterative discussions in the biology curriculum aimed at cultivating scientific citizenship. Encouragingly, students seemed excited to have a venue to bring up these questions, were genuinely interested in hearing different perspectives and the reasoning behind them and expressed eagerness to continue these discussions with family and friends.
Read the new "Teaching in Times of Crisis" issue of the Journal of Microbiology and Biology Education, including "Analysis of a COVID-19 Clinical Trial to Emphasize Experimental Design and Quantitative Reasoning in an Introductory Biology Course" by Melissa Grosser and Rebecca Hale.

Author: Melinda Grosser, Ph.D.

Melinda Grosser, Ph.D.
Dr. Melinda Grosser is an assistant professor in the Biology Department at the University of North Carolina Asheville.

Author: Rebecca Hale, Ph.D.

Rebecca Hale, Ph.D.
Dr. Rebecca Hale is an Associate Professor in the Biology Department at the University of North Carolina Asheville.