Comparative efficacy of BNT162b2 and mRNA-1273 vaccines in US veterans
Specification of target trials
We designed this observational analysis to emulate a target trial (i.e., a hypothetical pragmatic trial that would have answered the causal question of interest) of BNT162b2 versus mRNA-1273 for prevention of the outcomes of Covid-19 in the VA healthcare system. Key elements of the protocol are summarized in Table S1 of the Supplementary Appendix, available with the full text of this article on NEJM.org.
Eligibility criteria included veteran status, an age of at least 18 between January 4 and May 14, 2021, no previously documented SARS-CoV-2 infection, no previous vaccination against Covid- 19 and a known residential address outside of a long-term care facility, as well as known smoking status and recorded body mass index within the previous year. Participants were required to have used the VA healthcare system in the past year (defined as receiving care at a station eligible to administer study vaccines and having at least one primary care visit); however, they must have had no interaction with the health care system in the previous 3 days (which may have indicated the onset of symptomatic illness and prevented vaccination).
The interventions of interest were vaccination with the BNT162b2 vaccine or the mRNA-1273 vaccine, with a second dose scheduled 21 days later for the BNT162b2 vaccine and 28 days later for the mRNA-1273 vaccine. To ensure balance of important characteristics between the groups, eligible veterans in the target trial would be randomly assigned to one of these two vaccine groups in strata defined by calendar date (5-day bins ), age (5-year-old baccalaureate), sex (male or female), race (white, black, other or unknown), urbanity of residence (urban or non-urban) and geographic location ( coded as one of 19 categories of the Integrated Veterans Services Network).
The five outcomes of interest were documented SARS-CoV-2 infection, documented symptomatic Covid-19, hospital admission for Covid-19, ICU admission for Covid-19 and death due to Covid-19. For each eligible participant, follow-up began on the day the first vaccine dose was received (baseline) and ended on the day of the outcome of interest, death, 168 days (24 weeks) after baseline or end of study period (July 1, 2021), whichever comes first.
This target trial was designed to assess the comparative effectiveness of vaccines in a period when the alpha variant of SARS-CoV-2 was predominant. However, the alpha variant had declined to a 26% share of variants in circulation in the United States as of June 26, 2021, as it was quickly replaced by the delta variant, which fell from a 68% share as of July 3. 2021. , at 99% as of September 18, 2021.11 To assess the comparative efficacy of the vaccines in a delta-dominant period, we considered a second target trial that was identical to the first trial except that the enrollment period was July 1 to September 20, 2021, and the only outcome of interest was documented SARS-CoV-2 infection (because the period was too short to accumulate a sufficient number of rarer outcomes, such as hospitalization and death).
Emulation of target trials
We emulated the pragmatic target trials above using the VA healthcare databases, which are described in Supplementary Methods section 1 of the Supplementary Appendix. Table S2 provides detailed definitions of all study variables. The vaccination was identified using records from the Immunization domain and procedures recorded in the Ambulatory or Inpatient domain of the database. SARS-CoV-2 infections were identified using the national VA Covid-19 surveillance tool,12 which integrates polymerase chain reaction (PCR) laboratory test data with natural language processing of clinical notes to capture diagnoses inside and outside the VA healthcare system. Symptomatic Covid-19 was defined as at least one of the following symptoms documented in the VA healthcare system within 4 days before or after documentation of SARS-CoV-2 infection: fever, chills, cough , shortness of breath or difficulty breathing, sore throat, loss of taste or smell, headache, myalgia, diarrhea and vomiting. Symptoms were determined using records in the outpatient, inpatient, vital signs, health factors, and fee domains in the database. Hospitalization for Covid-19 was defined as hospitalization within 21 days of documentation of SARS-CoV-2 infection (confirmed with inpatient domain), ICU admission for Covid-19 was defined as ICU admission while hospitalized for Covid-19 (confirmed with Inpatient domain and specialty transfer codes), and Covid-19 death was defined as death within 30 days of documentation of SARS-CoV-2 infection (confirmed using the Patient domain).
To mimic the stratified randomization of the target trial, we matched eligible individuals who were vaccinated with BNT162b2 in a 1:1 ratio to eligible individuals who were vaccinated with mRNA-1273. Matching factors (calendar date, age, gender, race, urbanity of residence, and geographic location) are associated with the likelihood of receiving a particular vaccine, as well as the risk of SARS-CoV-2 infection or Severe Covid-19. (Additional details on the matching algorithm are provided in the Supplementary Methods section 2 of the Supplementary Appendix.)
To explore the possibility of residual confounding (e.g., by underlying health condition or healthcare-seeking behavior), we used two negative outcome controls that are not directly affected by vaccination. but for which the effect of vaccination could be confounded in the same way.13 First, we assessed the risk of symptomatic Covid-19 in the first 10 days after the first vaccine dose, during which no difference in risk between vaccines is expected.1.2 Second, we assessed the risk of death from causes other than Covid-19 during the follow-up period.
Covariate balance after matching was assessed by plotting the mean differences between variable values (standardized for continuous variables) for the vaccination groups, with a difference of 0.1 or less considered acceptable.14 The cumulative incidence (risk) curves for the vaccination groups were estimated with the Kaplan-Meier estimator.15 We considered the period from the day of the first dose of vaccine until the end of follow-up. We used the Kaplan-Meier estimator with daily outcome events to calculate the probability (risk) of the outcome over the period. We then calculated 24-week risk differences and relative risks between vaccination groups. We performed subgroup analyzes by age (
Analyzes were performed with R software, version 3.6.0 (R Foundation for Statistical Computing) and SAS software, version 8.2 (SAS Institute). Information about the authors’ contributions to the study is provided in Supplementary Methods Section 3 of the Supplementary Appendix. The first and last authors guarantee the accuracy and completeness of the data presented in this report.