covidVaxEffacacyBias_TruthForHealth
Vaccine

Outcome Reporting Bias in COVID-19 mRNA Vaccine Clinical Trials

by Ronald B. Brown

School of Public Health and Health Systems, University of Waterloo, Waterloo, ON N2L3G1, Canada

Academic Editor: Edgaras StankevičiusMedicina202157(3), 199; https://doi.org/10.3390/medicina57030199

Received: 13 January 2021 / Revised: 15 February 2021 / Accepted: 22 February 2021 / Published: 26 February 2021

Abstract

Relative risk reduction and absolute risk reduction measures in the evaluation of clinical trial data are poorly understood by health professionals and the public. The absence of reported absolute risk reduction in COVID-19 vaccine clinical trials can lead to outcome reporting bias that affects the interpretation of vaccine efficacy. The present article uses clinical epidemiologic tools to critically appraise reports of efficacy in Pfzier/BioNTech and Moderna COVID-19 mRNA vaccine clinical trials. Based on data reported by the manufacturer for Pfzier/BioNTech vaccine BNT162b2, this critical appraisal shows: relative risk reduction, 95.1%; 95% CI, 90.0% to 97.6%; p = 0.016; absolute risk reduction, 0.7%; 95% CI, 0.59% to 0.83%; p < 0.000. For the Moderna vaccine mRNA-1273, the appraisal shows: relative risk reduction, 94.1%; 95% CI, 89.1% to 96.8%; p = 0.004; absolute risk reduction, 1.1%; 95% CI, 0.97% to 1.32%; p < 0.000. Unreported absolute risk reduction measures of 0.7% and 1.1% for the Pfzier/BioNTech and Moderna vaccines, respectively, are very much lower than the reported relative risk reduction measures. Reporting absolute risk reduction measures is essential to prevent outcome reporting bias in evaluation of COVID-19 vaccine efficacy.Keywords: mRNA vaccineCOVID-19 vaccinevaccine efficacyrelative risk reductionabsolute risk reductionnumber needed to vaccinateoutcome reporting biasclinical epidemiologycritical appraisalevidence-based medicine

1. Introduction

Using messenger RNA (mRNA) in vaccines to produce proteins that trigger an immune response against infectious diseases has held promise for decades, but until recently, no clinically tested mRNA vaccine has managed to advance beyond small, early-phase trials [1]. Normally, genetic code in mRNA is transcribed from DNA in the cell nucleus, and the coded message is delivered by mRNA to cell ribosomes for translation during protein biosynthesis [2]. COVID-19 mRNA vaccines directly inject cells with a synthetic genetic code to replicate the spike S protein found on the surface of the coronavirus, SARS-CoV-2 [3]. Once replicated, the spike protein is proposed to trigger an immune response that creates antibodies against the virus [4].However, several biological obstacles continue to challenge the development of mRNA vaccines, including “mRNA’s extremely large size, charge, intrinsic instability, and high susceptibility to enzymatic degradation” [5]. To mitigate enzymatic degradation, mRNA in the vaccines is encapsulated in lipid nanoparticles [6], but it is unclear how this encapsulation affects genetic code translation in the cell ribosomes. Nevertheless, clinical results of phase III trials reported for COVID-19 vaccines manufactured by Pfizer/BioNTech (New York City, NY, USA/Mainz, Germany) [7] and Moderna (Cambridge, MA, USA) [8] have far surpassed predicted performance, with vaccine efficacy rates of approximately 95%. Curiously, “why these vaccines seem so effective while previous attempts against other pathogens haven’t appeared as promising remains an open question” [1].As noted in BMJ Opinion, 26 November 2020 [9],“There may be much more complexity to the ‘95% effective’ announcement than meets the eye—or perhaps not. Only full transparency and rigorous scrutiny of the data will allow for informed decision making. The data must be made public.”As was also noted in the BMJ Opinion, Pfizer/BioNTech and Moderna reported the relative risk reduction of their vaccines, but the manufacturers did not report a corresponding absolute risk reduction, which “appears to be less than 1%” [9]. Absolute risk reduction (ARR) and relative risk reduction (RRR) are measures of treatment efficacy reported in randomized clinical trials. Because the ARR and RRR can be dramatically different in the same trial, it is necessary to include both measures when reporting efficacy outcomes to avoid outcome reporting bias. In the present article, a critical appraisal of publicly available clinical trial data verifies that absolute risk reduction percentages for Pfizer/BioNTech vaccine BNT162b2 [7] and Moderna vaccine mRNA-1273 [8] are, respectively, 0.7%; 95% CI, 0.59% to 0.83%; p = 0.000, and 1.1%; 95% CI, 0.97% to 1.32%; p = 0.000. The same publicly available data, without absolute risk reduction measures, were reviewed and approved by the roster of members serving on the U.S. Food and Drug Administration’s (FDA’s) Vaccines and Related Biological Products Advisory Committee (VRBPAC) for emergency use authorization (EUA) of the mRNA vaccines [10]. Ironically, the omission of absolute risk reduction measures in data reviewed by the VRBPAC overlooks FDA guidelines for communicating evidence-based risks and benefits to the public [11]. The FDA’s advice for information providers includes:“Provide absolute risks, not just relative risks. Patients are unduly influenced when risk information is presented using a relative risk approach; this can result in suboptimal decisions. Thus, an absolute risk format should be used.”The New England Journal of Medicine also published clinical trial data on safety and efficacy for the BNT162b2 vaccine [12] and the mRNA-1273 vaccine [13], but with no mention of absolute risk reduction measures.The present article uses epidemiologic tools to critically appraise absolute and relative risk reduction measures for vaccine efficacy in phase III clinical trials of the COVID-19 mRNA vaccines. Microsoft Excel was used to analyze data and chart risk reduction outcomes. The article further clarifies how selective reporting of vaccine efficacy measures can cause a type of outcome reporting bias that misrepresents health information disseminated to the public.

2. Critical Appraisal of Vaccine Efficacy

The application of epidemiologic and biometric methods to clinical diagnosis and treatment is known as clinical epidemiology [14]. Clinical epidemiologic tools can be applied in evidence-based medicine (EBM) to critically appraise research evidence for validity, size of effect, and usefulness in clinical practice [15]. Clinical treatment effects in groups of participants are measured by comparing probabilities of an event, known as event rates [16].Figure 1 shows an example of a vaccine clinical trial for an infectious disease. The vaccine and placebo groups in Figure 1 each have 100 randomly assigned individuals with no history of infection, and an event is defined as the incidence of infection among all individuals during the course of the trial. The percentage of events in the vaccine group is the experimental event rate (EER) or the risk of infection in the vaccine group (1/100 = 1%), and the percentage of events in the placebo group is the control event rate (CER) or the risk of infection in the placebo group (2/100 = 2%). Absolute risk reduction (ARR) is the disease risk difference between the placebo and vaccine groups, i.e., the CER minus the EER (2% − 1% = 1%). The ARR is also known as the vaccine disease preventable incidence (VDPI) [17]. Relative risk reduction (RRR) or vaccine efficacy (VE) is the reduced risk from vaccination, the ARR or VDPI, relative to or divided by the risk in unvaccinated individuals, the CER (1%/2% = 50%) [18].

Medicina 57 00199 g001 550

Figure 1. Example of a vaccine clinical trial for an infectious disease.

3. 2 × 2 Contingency Tables and Epidemiologic Equations

The following 2 × 2 contingency tables for SARS-CoV-2 infection are based on reported clinical trial data for the Pfzier/BioNTech BNT162b2 vaccine [12] and the Moderna mRNA-1273 vaccine [13]. The table rows, shown in Table 1, list the vaccine and placebo groups and the table columns list the participants’ outcomes of either SARS-CoV-2 infection or no infection. Table 2 and Table 3 list the clinical trial data for the Pfzier/BioNTech and Moderna vaccines, respectively. As shown in Table 1, the total number of participants in a group, known as n, is represented by a + b for the vaccine group and c + d for the placebo group.Table 1. 2 × 2 contingency table for SARS-CoV-2 infection in vaccine clinical trials.

Table

Table 2. 2 × 2 contingency table for SARS-CoV-2 infection in Pfzier/BioNTech vaccine clinical trial.

Table

Table 3. 2 × 2 contingency table for SARS-CoV-2 infection in Moderna vaccine clinical trial.

Table

The following epidemiologic equations use data from the 2 × 2 contingency tables (Table 1Table 2 and Table 3) to calculate relative and absolute measures of COVID-19 mRNA vaccine efficacy.Risk ratio (RR): RR = a/(a+b)c/(c+d)(1)The risk ratio, also known as the relative risk, in a randomized controlled trial is the ratio calculated by dividing the experimental event rate (EER), a/(a + b), by the control event rate (CER), c/(c + d) [19]. Dividing the EER by the CER equals 1 if the rates do not differ, in which case the RR has the null value 1. RRs below 1 indicate a protective effect and a decreased risk (EER < CER), and RRs above 1 indicate an increased risk (EER > CER).Risk ratio 95% confidence interval (CI): CI = e^(Ln(RR) ± 1.96∗SE) where SE = b/a(a+b)+d/c(c+d)−−−−−−−−−−−−−−√ or 1a−1(a+b)+1c−1(c+d) −−−−−−−−−−−−−−−−−−−−−−√ (2)The risk ratio 95% confidence interval predicts the range of probable risk ratios if the experiment or trial was repeated 95 out of 100 times. The narrower the range between the upper and lower CI values, the more precise the CI. If the range includes the RR null value, 1, the risk ratio is considered statistically insignificant. The equation calculates the standard error (SE) [20,21], and the natural logarithm (Ln) is used, along with the antilog expressed as an exponent of the base e, to normally distribute the data when calculating the 95% probability.Absolute risk reduction (ARR):ARR (%) = c(c+d)− a(a+b)(3)The absolute risk reduction is a percentage equal to the arithmetic difference when subtracting the EER from the CER [19]. The difference equals zero if the rates do not differ, in which case the ARR has the null value zero. The difference is negative if the EER is higher than the CER.Absolute risk reduction 95% confidence interval (CI upper, lower):ARR CI = ARR ± 1.96∗SE, where SE = EER∗(1 − EER)(a+b) + CER∗(1 − CER)(c+d)−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−√(4)The standard error in the absolute risk reduction 95% confidence interval measures the square root of the sum of the group variances [22]. If the ARR CI includes the null value zero, the ARR is not statistically significant.Number needed to vaccinate (NNV):NNV = 1ARR(5)The NNV, or the number needed to vaccinate to prevent one infection, is the reciprocal of the ARR [17]. Note that the numerator is multiplied by 100 when the ARR is expressed with a percentage sign. The NNV is also usually rounded up to the next individual.NNV 95% confidence interval (CI):NNV CI = 1ARR CI (6)The CI of the NNV is calculated by dividing 1 by the ARR CI [22], again multiplying by 100 in the numerator when the ARR is expressed with a percentage sign.Relative risk reduction (RRR) or vaccine efficacy (VE):RRR, VE (%) = 1 − RR(7)The relative risk reduction is the same as vaccine efficacy (VE) [17]. The RRR is calculated by subtracting the RR from the null value 1, or by dividing the ARR by the CER [22].RRR, VE 95% confidence interval (CI):RRR, VE CI = 1 − RR CI(8)The CI for the relative risk reduction is calculated by subtracting the RR CI from the null value 1.Pvalues, which measure the probability that a trial result occurred by chance, can be calculated from the confidence interval for the difference between two proportions, as in the ARR, and from the confidence interval for a ratio, as in the RRR [23]. Online calculators are also available that compare group proportions [24] and calculate epidemiological equations [25], which are useful for measuring vaccine efficacy. Figure 2 shows a chart of the present critical appraisal of mRNA COVID-19 vaccine efficacy. Note that the vertical axis of the chart is a logarithmic scale, base 10.

Medicina 57 00199 g002 550

Figure 2. The chart shows critical appraisal results of mRNA COVID-19 vaccine efficacy.Clinical epidemiologic tools can be used to critically appraise the efficacy of new COVID-19 vaccines having biological mechanisms that differ from the mRNA vaccines, such as AstraZeneca-Oxford’s ChAdOx1 adenoviral vector vaccine [26] and Johnson & Johnson’s Janssen Biotech Ad26.COV2.S vaccine [27]. (As this article goes to press, the FDA VRBPAC is scheduled to review the Janssen Biotech vaccine for EUA.) As well, reported efficacy for randomized clinical trials involving any treatment, intervention, disease, disorder, or illness can be critically appraised using clinical epidemiologic tools. In a similar manner, observational studies that report vaccine and other treatment effectiveness in reducing disease incidence within a population can also be critically appraised using clinical epidemiologic tools.

4. Discussion

Medical and public health experts continue to stress the need to include measurements of absolute risk reduction and number needed to treat when reporting results of clinical interventions [28]. Currently, differences between relative effect measures and absolute effect measures in studies are “poorly understood by health professionals, and even more poorly understood by patients.” [29] In addition,“…critical appraisal knowledge and skills are limited among physicians,” and “use of relative effect measures was associated with greater perceptions of medication effectiveness and intent to prescribe, compared with the use of absolute effect measures.”[29]Reporting relative measures may be sufficient to summarize evidence of a study for comparisons with other studies, but absolute measures are also necessary for applying study findings to specific clinical or public health circumstances [22]. Omitting absolute risk reduction findings in public health and clinical reports of vaccine efficacy is an example of outcome reporting bias, which ignores unfavorable outcomes and misleads the public’s impression and scientific understanding of a treatment’s efficacy and benefits [30]. Furthermore, the ethical and legal obligation of informed consent requires that patients are educated about the risks and benefits of a healthcare procedure or intervention [31].Similar to the critical appraisal in the present article, critical appraisals of reported vaccine efficacy in other studies reveals clinically significant insights. For example, a 2018 review of 52 randomized trials for influenza vaccines that studied over 80,000 healthy adults reported an overall influenza vaccine EER of 0.9% and a 2.3% CER, which calculates to a RRR of 60.8% [32]. This vaccine efficacy is consistent with a 40% to 60% reduction in influenza reported by the Centers for Disease Control and Prevention (CDC) [33]. However, critically appraising data from the 2018 review shows an overall ARR of only 1.4%, which reveals vital clinical information that is missing in the CDC report. A 1.4% ARR works out to a NNV of approximately 72 people, meaning that 72 individuals need to be vaccinated to reduce one case of influenza. By comparison, Figure 2 of the present article shows that the NNV for the Pfzier-BioNTech and Moderna vaccines are 142 (95% CI 122 to 170) and 88 (95% CI 76 to 104), respectively.The mRNA vaccine manufacturers reported that infections in most subgroups in phase III clinical trials were similar for both vaccines after two doses. Vaccine clinical trial case definitions for SARS-CoV-2 infection included COVID-19 clinical symptoms; thus the trials were not designed to provide evidence of vaccine efficacy for protection against asymptomatic infections. In addition to outcome reporting bias, information bias may have also affected COVID-19 vaccine trial outcomes due to misclassification of SARS-CoV-2 infections as mild adverse effects of the vaccines. For example, several COVID-19 clinical symptoms are similar to the vaccines’ adverse effects such as fever, pain, and fatigue, which could potentially lead to missed diagnoses of viral infections.A limitation of this article is that it only critically appraised mRNA vaccine efficacy in healthy individuals who were randomized to two groups under strictly controlled conditions. The critical appraisal did not include vaccine safety and effectiveness outcomes within a general population that includes unhealthy people and that lacks control over confounding factors. For example, healthy vaccinee bias occurs when people who are in better health are more likely to follow vaccination recommendations in order to protect their health [34].

5. Conclusions

A critical appraisal of phase III clinical trial data for the Pfizer/BioNTech vaccine BNT162b2 and Moderna vaccine mRNA-1273 shows that absolute risk reduction measures are very much lower than the reported relative risk reduction measures. Yet, the manufacturers failed to report absolute risk reduction measures in publicly released documents. As well, the U.S FDA Advisory Committee (VRBPAC) did not follow FDA published guidelines for communicating risks and benefits to the public, and the committee failed to report absolute risk reduction measures in authorizing the BNT162b2 and mRNA-1273 vaccines for emergency use. Such examples of outcome reporting bias mislead and distort the public’s interpretation of COVID-19 mRNA vaccine efficacy and violate the ethical and legal obligations of informed consent.

Funding

This research received no external funding.

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

Data for Pfzier/BioNTech BNT162b2: https://doi.org/10.1056/nejmoa2034577; data for Moderna mRNA-1273: https://doi.org/10.1056/NEJMoa2035389 (accessed on 10 January 2021).

Acknowledgments

Special thanks to Richard J. Cook from the University of Waterloo, whose foundational work with David L. Sackett in clinical epidemiology is cited in this manuscript.

Conflicts of Interest

The author declares no conflict of interest.

References

  1. Kwon, D. The Promise of mRNA Vaccines. Available online: https://www.the-scientist.com/news-opinion/the-promise-of-mrna-vaccines-68202 (accessed on 23 December 2020).
  2. Genome. Messenger RNA (mRNA). Available online: https://www.genome.gov/genetics-glossary/messenger-rna (accessed on 23 December 2020).
  3. Garde, D. The Story of mRNA: How a Once-Dismissed Idea Became a Leading Technology in the Covid Vaccine Race. Available online: https://www.statnews.com/2020/11/10/the-story-of-mrna-how-a-once-dismissed-idea-became-a-leading-technology-in-the-covid-vaccine-race/ (accessed on 5 January 2021).
  4. Centers for Disease Control and Prevention. Understanding mRNA COVID-19 Vaccines. Available online: https://www.cdc.gov/coronavirus/2019-ncov/vaccines/different-vaccines/mrna.html (accessed on 22 December 2020).
  5. Wadhwa, A.; Aljabbari, A.; Lokras, A.; Foged, C.; Thakur, A. Opportunities and Challenges in the Delivery of mRNA-based Vaccines. Pharmaceutics 202012, 102. [Google Scholar] [CrossRef] [PubMed]
  6. Reichmuth, A.M.; Oberli, M.A.; Jaklenec, A.; Langer, R.; Blankschtein, D. mRNA vaccine delivery using lipid nanoparticles. Ther. Deliv. 20167, 319–334. [Google Scholar] [CrossRef] [PubMed]
  7. Food and Drug Administration. Pfizer-BioNTech COVID-19 Vaccine VRBPAC Briefing Document. Available online: https://www.fda.gov/media/144246/download (accessed on 23 December 2020).
  8. Food and Drug Administration. FDA Briefing Document: Moderna COVID-19 Vaccine. Available online: https://www.fda.gov/media/144434/download (accessed on 23 December 2020).
  9. Doshi, P. Peter Doshi: Pfizer and Moderna’s ”95% effective” Vaccines—Let’s Be Cautious and First See the Full Data. Available online: https://blogs.bmj.com/bmj/2020/11/26/peter-doshi-pfizer-and-modernas-95-effective-vaccines-lets-be-cautious-and-first-see-the-full-data/ (accessed on 23 December 2020).
  10. Food and Drug Administration. Roster of the Vaccines and Related Biological Products Advisory Committee. Available online: https://www.fda.gov/advisory-committees/vaccines-and-related-biological-products-advisory-committee/roster-vaccines-and-related-biological-products-advisory-committee (accessed on 23 December 2020).
  11. Fischhoff, B.; Brewer, N.; Downs, J. Communicating Risks and Benefits: An Evidence-Based User’s Guide; Food and Drug Administration (FDA), US Department of Health and Human Services: Silver Spring, MA, USA, 2011.
  12. Polack, F.P.; Thomas, S.J.; Kitchin, N.; Absalon, J.; Gurtman, A.; Lockhart, S.; Perez, J.L.; Pérez Marc, G.; Moreira, E.D.; Zerbini, C. Safety and efficacy of the BNT162b2 mRNA covid-19 vaccine. N. Engl. J. Med. 2020383. [Google Scholar] [CrossRef] [PubMed]
  13. Baden, L.R.; El Sahly, H.M.; Essink, B.; Kotloff, K.; Frey, S.; Novak, R.; Diemert, D.; Spector, S.A.; Rouphael, N.; Creech, C.B.; et al. Efficacy and safety of the mRNA-1273 SARS-CoV-2 vaccine. N. Engl. J. Med. 2020384. [Google Scholar] [CrossRef]
  14. Sackett, D.L. Clinical epidemiology. Am. J. Epidemiol. 196989, 125–128. [Google Scholar] [CrossRef] [PubMed]
  15. Sackett, D.; Straus, S.; Scott Richardson, W.; Rosenberg, W.; Haynes, R. Evidence-Based Medicine: How to Practice and Teach EBM, 2nd ed.; Churchill Livingstone: Edinburgh, UK; London, UK, 2000. [Google Scholar]
  16. Cook, R.J.; Sackett, D.L. The number needed to treat: A clinically useful measure of treatment effect. BMJ 1995310, 452–454. [Google Scholar] [CrossRef] [PubMed]
  17. Dasgupta, S. A Review of Vaccine Efficacy Measures. Vaccin Res. Open J. 20191, 61–64. [Google Scholar]
  18. Irwig, L.; Irwig, J.; Revena, L.; Sweet, M. Relative risk, relative and absolute risk reduction, number needed to treat and confidence intervals. In Smart Health Choices: Making Sense of Health Advice; Hammersmith Press: London, UK, 2008; Chapter 18. [Google Scholar]
  19. Kremer, L.; Moyer, V. Tips and tricks for understanding and using SR results—No 1: Relative risk, risk difference, and number needed to treat. Evidence-Based Child Health Cochrane Rev. J. 20094, 1146–1148. [Google Scholar] [CrossRef]
  20. Sullivan, L. Confidence Intervals for the Risk Ratio (Relative Risk). Available online: https://sphweb.bumc.bu.edu/otlt/mph-modules/bs/bs704_confidence_intervals/bs704_confidence_intervals8.html#:~:text=Therefore%2C%20computing%20the%20confidence%20interval,confidence%20interval%20for%20the%20RR (accessed on 26 December 2020).
  21. Morris, J.A.; Gardner, M.J. Calculating Confidence Intervals For Relative Risks (Odds Ratios) And Standardised Ratios And Rates. BMJ 1988296, 1313–1316. [Google Scholar] [CrossRef] [PubMed]
  22. Schechtman, E. Odds ratio, relative risk, absolute risk reduction, and the number needed to treat—Which of these should we use? Value Health 20025, 431–436. [Google Scholar] [CrossRef] [PubMed]
  23. Altman, D.G.; Bland, J.M. How to obtain the P value from a confidence interval. BMJ 2011343, d2304. [Google Scholar] [CrossRef] [PubMed]
  24. Azzopardi, D. Group Comparison Calculator. Available online: https://www.neoweb.org.uk/Additions/compare.htm (accessed on 6 January 2021).
  25. Heidel, E. Epidemiology. Available online: https://www.scalestatistics.com/epidemiology.html (accessed on 6 January 2021).
  26. Voysey, M.; Clemens, S.A.C.; Madhi, S.A.; Weckx, L.Y.; Folegatti, P.M.; Aley, P.K.; Angus, B.; Baillie, V.L.; Barnabas, S.L.; Bhorat, Q.E. Safety and Efficacy of the ChAdOx1 nCoV-19 Vaccine (AZD1222) against SARS-CoV-2: An Interim Analysis of Four Randomised Controlled Trials in Brazil, South Africa, and the UK. Lancet 2021397, 99–111. [Google Scholar] [CrossRef]
  27. Sadoff, J.; Le Gars, M.; Shukarev, G.; Heerwegh, D.; Truyers, C.; de Groot, A.M.; Stoop, J.; Tete, S.; Van Damme, W.; Leroux-Roels, I. Interim Results of a Phase 1–2a Trial of Ad26. COV2. S Covid-19 Vaccine. N. Engl. J. Med. 2021. [Google Scholar] [CrossRef] [PubMed]
  28. Elliott, M.H.; Skydel, J.J.; Dhruva, S.S.; Ross, J.S.; Wallach, J.D. Characteristics and Reporting of Number Needed to Treat, Number Needed to Harm, and Absolute Risk Reduction in Controlled Clinical Trials, 2001–2019. JAMA Intern. Med. 2020181. [Google Scholar]
  29. Kahwati, L.; Carmody, D.; Berkman, N.; Sullivan, H.W.; Aikin, K.J.; DeFrank, J. Prescribers’ Knowledge and Skills for Interpreting Research Results: A Systematic Review. J. Contin. Educ. Health Prof. 201737, 129–136. [Google Scholar] [CrossRef]
  30. Thomas, E.T.; Heneghan, C. Outcome Reporting Bias. Available online: https://catalogofbias.org/biases/outcome-reporting-bias/ (accessed on 23 November 2020).
  31. Shah, P.; Thornton, I.; Turrin, D.; Hipskind, J.E. Informed Consent. Available online: https://www.ncbi.nlm.nih.gov/books/NBK430827/#:~:text=Informed%20consent%20is%20the%20process,undergo%20the%20procedure%20or%20intervention (accessed on 8 February 2021).
  32. Demicheli, V.; Jefferson, T.; Ferroni, E.; Rivetti, A.; Di Pietrantonj, C. Vaccines for preventing influenza in healthy adults. In Cochrane Database Of Systematic Reviews; Cochrane Library: Hoboken, NJ, USA, 2018. Available online: https://pubmed.ncbi.nlm.nih.gov/29388196 (accessed on 7 January 2021).
  33. Centers for Disease Control and Prevention. Vaccine Effectiveness: How Well Do the Flu Vaccines Work? Available online: https://www.cdc.gov/flu/vaccines-work/vaccineeffect.htm (accessed on 26 November 2020).
  34. Remschmidt, C.; Wichmann, O.; Harder, T. Frequency and impact of confounding by indication and healthy vaccinee bias in observational studies assessing influenza vaccine effectiveness: A systematic review. BMC Infect. Dis. 201515, 429. [Google Scholar] [CrossRef] [PubMed]
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

© 2021 by the author. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

You may also like...

Popular Articles...

106 Comments

  1. You’re so awesome! I don’t believe I have read a single thing like that before. So great to find someone with some original thoughts on this topic. Really.. thank you for starting this up. This website is something that is needed on the internet, someone with a little originality!

  2. I¦ll right away clutch your rss feed as I can not find your e-mail subscription link or newsletter service. Do you have any? Kindly let me recognise in order that I may just subscribe. Thanks.

  3. Thanks for sharing superb informations. Your web site is so cool. I’m impressed by the details that you have on this site. It reveals how nicely you perceive this subject. Bookmarked this website page, will come back for extra articles. You, my pal, ROCK! I found simply the information I already searched all over the place and simply could not come across. What a perfect web site.

  4. Hey There. I discovered your blog the use of msn. That is an extremely smartly written article. I will be sure to bookmark it and return to learn more of your helpful information. Thanks for the post. I will definitely return.

  5. I have recently started a web site, the info you provide on this website has helped me greatly. Thank you for all of your time & work.

  6. Magnificent beat ! I wish to apprentice while you amend your web site, how could i subscribe for a blog site? The account helped me a acceptable deal. I had been a little bit acquainted of this your broadcast provided bright clear idea

  7. Spot on with this write-up, I truly think this website needs much more consideration. I’ll most likely be again to learn rather more, thanks for that info.

  8. I loved up to you’ll obtain performed proper here. The caricature is attractive, your authored subject matter stylish. however, you command get bought an impatience over that you want be delivering the following. ill indubitably come further earlier again as precisely the similar just about a lot regularly inside of case you shield this hike.

  9. Outstanding post, you have pointed out some superb details , I likewise think this s a very wonderful website.

  10. Great ?V I should certainly pronounce, impressed with your website. I had no trouble navigating through all tabs and related information ended up being truly easy to do to access. I recently found what I hoped for before you know it in the least. Quite unusual. Is likely to appreciate it for those who add forums or something, website theme . a tones way for your customer to communicate. Excellent task..

  11. I like what you guys are up too. Such smart work and reporting! Carry on the excellent works guys I have incorporated you guys to my blogroll. I think it will improve the value of my web site 🙂

  12. Yeah bookmaking this wasn’t a speculative conclusion outstanding post! .

  13. It?¦s in point of fact a nice and helpful piece of info. I?¦m happy that you shared this useful information with us. Please stay us informed like this. Thank you for sharing.

  14. Hello, Neat post. There is a problem with your web site in web explorer, could test thisK IE nonetheless is the market leader and a huge portion of other people will miss your fantastic writing because of this problem.

  15. An interesting discussion is worth comment. I think that you should write more on this topic, it might not be a taboo subject but generally people are not enough to speak on such topics. To the next. Cheers

  16. This blog is definitely rather handy since I’m at the moment creating an internet floral website – although I am only starting out therefore it’s really fairly small, nothing like this site. Can link to a few of the posts here as they are quite. Thanks much. Zoey Olsen

  17. I am really impressed with your writing skills as well as with the layout on your blog. Is this a paid theme or did you customize it yourself? Anyway keep up the excellent quality writing, it’s rare to see a great blog like this one nowadays..

  18. I’m usually to blogging and i really appreciate your content. The article has really peaks my interest. I’m going to bookmark your website and preserve checking for new information.

  19. After all, what a great site and informative posts, I will upload inbound link – bookmark this web site? Regards, Reader.

  20. I like this web site so much, saved to fav. “I don’t care what is written about me so long as it isn’t true.” by Dorothy Parker.

  21. I like this site very much, Its a rattling nice situation to read and incur info . “Education is the best provision for old age.” by Aristotle.

  22. I genuinely enjoy examining on this internet site, it has wonderful content.

  23. you have a great blog here! would you like to make some invite posts on my blog?

  24. Great blog here! Also your web site loads up fast! What host are you using? Can I get your affiliate link to your host? I wish my site loaded up as quickly as yours lol

  25. I am thankful that I detected this weblog, just the right info that I was searching for! .

  26. There is evidently a lot to identify about this. I feel you made some good points in features also.

  27. Your style is so unique compared to many other people. Thank you for publishing when you have the opportunity,Guess I will just make this bookmarked.2

  28. Does your blog have a contact page? I’m having trouble locating it but, I’d like to shoot you an email. I’ve got some recommendations for your blog you might be interested in hearing. Either way, great site and I look forward to seeing it grow over time.

  29. Thanks for this terrific post, I am glad I observed this internet site on yahoo.

  30. As I website possessor I believe the subject matter here is real good, thanks for your efforts.

  31. As a Newbie, I am continuously searching online for articles that can help me. Thank you

  32. I’ve been absent for a while, but now I remember why I used to love this site. Thanks, I’ll try and check back more frequently. How frequently you update your site?

  33. Thanks for sharing superb informations. Your website is so cool. I am impressed by the details that you¦ve on this web site. It reveals how nicely you understand this subject. Bookmarked this web page, will come back for more articles. You, my pal, ROCK! I found simply the information I already searched everywhere and just couldn’t come across. What an ideal web site.

  34. Valuable info. Lucky me I found your web site by accident, and I’m shocked why this accident did not happened earlier! I bookmarked it.

  35. I believe this website has got very good composed articles articles.

  36. There may be noticeably a bundle to know about this. I assume you made sure good points in features also.

  37. You have brought up a very superb points, thankyou for the post.

  38. There is evidently a bunch to know about this. I think you made various good points in features also.

  39. Hi my friend! I wish to say that this article is amazing, great written and include almost all significant infos. I would like to peer more posts like this .

  40. Hello my family member! I want to say that this post is amazing, great written and come with almost all important infos. I would like to look more posts like this .

  41. Thank you a lot for giving everyone an extraordinarily marvellous chance to read from this web site. It is usually so terrific and as well , packed with a great time for me and my office friends to visit your website no less than 3 times in a week to read the fresh tips you have got. And of course, we are always contented concerning the beautiful techniques served by you. Certain 4 ideas on this page are easily the finest we have had.

  42. I reckon something genuinely interesting about your weblog so I saved to fav.

  43. I always was concerned in this topic and stock still am, regards for posting.

  44. There is noticeably a bundle to know about this. I assume you made certain good factors in features also.

  45. Hello.This post was really motivating, particularly because I was looking for thoughts on this matter last Thursday.

  46. Well I sincerely liked studying it. This subject procured by you is very useful for good planning.

  47. I have read a few just right stuff here. Definitely worth bookmarking for revisiting. I surprise how much attempt you place to make such a fantastic informative web site.

  48. Great selection of modern and classic books waiting to be discovered. All free and available in most ereader formats. download free books

  49. I’d have to examine with you here. Which is not one thing I usually do! I take pleasure in reading a post that may make folks think. Additionally, thanks for permitting me to comment!

  50. Whats up are using WordPress for your site platform? I’m new to the blog world but I’m trying to get started and create my own. Do you need any coding knowledge to make your own blog? Any help would be really appreciated!

  51. I have fun with, cause I found just what I used to be looking for. You’ve ended my four day lengthy hunt! God Bless you man. Have a nice day. Bye

  52. Needed to put you the tiny note so as to thank you so much yet again about the nice things you’ve discussed on this website. This is certainly tremendously generous with people like you to convey unhampered all a lot of people would’ve sold for an ebook in order to make some profit on their own, most notably considering the fact that you could have done it in case you wanted. The things additionally worked to provide a good way to fully grasp many people have the identical keenness like my own to know a whole lot more regarding this issue. I am sure there are numerous more pleasurable instances up front for folks who read carefully your website.

  53. whoah this blog is wonderful i really like reading your articles. Keep up the great paintings! You realize, a lot of people are hunting round for this info, you could help them greatly.

  54. I think other website proprietors should take this site as an model, very clean and fantastic user genial style and design, as well as the content. You are an expert in this topic!

  55. Greetings! I’ve been following your site for a long time now and finally got the bravery to go ahead and give you a shout out from Porter Tx! Just wanted to say keep up the great work!

  56. I was recommended this website by my cousin. I am not sure whether this post is written by him as nobody else know such detailed about my trouble. You’re incredible! Thanks!

  57. Usually I don’t read post on blogs, but I would like to say that this write-up very forced me to try and do it! Your writing style has been surprised me. Thanks, quite nice article.

  58. Thanks for this terrific post, I am glad I discovered this website on yahoo.

  59. Some genuinely interesting points you have written.Assisted me a lot, just what I was searching for : D.

  60. Appreciating the commitment you put into your website and detailed information you present. It’s great to come across a blog every once in a while that isn’t the same old rehashed material. Fantastic read! I’ve bookmarked your site and I’m including your RSS feeds to my Google account.

  61. I discovered your blog site on google and check a few of your early posts. Continue to keep up the very good operate. I just additional up your RSS feed to my MSN News Reader. Seeking forward to reading more from you later on!…

  62. Hi! I just wanted to ask if you ever have any problems with hackers? My last blog (wordpress) was hacked and I ended up losing a few months of hard work due to no back up. Do you have any solutions to prevent hackers?

  63. I rattling thankful to find this internet site on bing, just what I was searching for : D besides saved to favorites.

  64. Glad to be one of several visitors on this awe inspiring site : D.

  65. Heya i’m for the first time here. I found this board and I find It really useful & it helped me out a lot. I hope to give something back and help others like you aided me.

  66. An interesting discussion is worth comment. I think that you should write more on this topic, it might not be a taboo subject but generally people are not enough to speak on such topics. To the next. Cheers

  67. I was suggested this web site by my cousin. I am not sure whether this post is written by him as no one else know such detailed about my trouble. You’re amazing! Thanks!

  68. I am constantly thought about this, thankyou for posting.

  69. I think this web site holds some real good information for everyone : D.

  70. Howdy just wanted to give you a quick heads up. The text in your post seem to be running off the screen in Firefox. I’m not sure if this is a format issue or something to do with internet browser compatibility but I thought I’d post to let you know. The layout look great though! Hope you get the issue solved soon. Cheers

  71. My coder is trying to convince me to move to .net from PHP. I have always disliked the idea because of the expenses. But he’s tryiong none the less. I’ve been using Movable-type on numerous websites for about a year and am worried about switching to another platform. I have heard very good things about blogengine.net. Is there a way I can import all my wordpress content into it? Any kind of help would be greatly appreciated!

  72. What’s Happening i am new to this, I stumbled upon this I have found It positively useful and it has helped me out loads. I hope to contribute & aid other users like its aided me. Great job.

  73. When I originally commented I clicked the -Notify me when new comments are added- checkbox and now each time a comment is added I get four emails with the same comment. Is there any way you can remove me from that service? Thanks!

  74. I have not checked in here for some time as I thought it was getting boring, but the last several posts are good quality so I guess I?¦ll add you back to my everyday bloglist. You deserve it my friend 🙂

  75. Some genuinely interesting details you have written.Helped me a lot, just what I was searching for : D.

  76. Wow that was odd. I just wrote an very long comment but after I clicked submit my comment didn’t appear. Grrrr… well I’m not writing all that over again. Regardless, just wanted to say fantastic blog!

  77. Hi there! I’m at work surfing around your blog from my new iphone! Just wanted to say I love reading your blog and look forward to all your posts! Keep up the great work!

  78. Some times its a pain in the ass to read what website owners wrote but this site is real user pleasant! .

  79. Woah! I’m really digging the template/theme of this website. It’s simple, yet effective. A lot of times it’s very hard to get that “perfect balance” between user friendliness and visual appearance. I must say you’ve done a fantastic job with this. Additionally, the blog loads very quick for me on Internet explorer. Excellent Blog!

  80. Some really choice posts on this web site, saved to my bookmarks.

  81. There is noticeably a bundle to find out about this. I assume you made sure good factors in features also.

  82. Hi , I do believe this is an excellent blog. I stumbled upon it on Yahoo , i will come back once again. Money and freedom is the best way to change, may you be rich and help other people.

  83. I’m not sure where you’re getting your info, but great topic. I needs to spend some time learning much more or understanding more. Thanks for excellent info I was looking for this info for my mission.

  84. Appreciate it for all your efforts that you have put in this. very interesting info .

  85. Your place is valueble for me. Thanks!…

  86. You made some good points there. I did a search on the subject and found most individuals will agree with your site.

  87. Whats up this is kind of of off topic but I was wanting to know if blogs use WYSIWYG editors or if you have to manually code with HTML. I’m starting a blog soon but have no coding experience so I wanted to get guidance from someone with experience. Any help would be enormously appreciated!

  88. Hi, just required you to know I he added your site to my Google bookmarks due to your layout. But seriously, I believe your internet site has 1 in the freshest theme I??ve came across. It extremely helps make reading your blog significantly easier.

  89. I?¦ll immediately grasp your rss feed as I can not find your e-mail subscription hyperlink or e-newsletter service. Do you have any? Kindly allow me recognise in order that I may just subscribe. Thanks.

  90. You are my inspiration , I own few web logs and infrequently run out from to brand.

  91. What¦s Happening i’m new to this, I stumbled upon this I have discovered It positively useful and it has helped me out loads. I’m hoping to contribute & aid different users like its aided me. Good job.

  92. whoah this blog is wonderful i really like reading your articles. Keep up the great paintings! You realize, a lot of people are hunting round for this info, you could help them greatly.

  93. I have read so many posts about the blogger lovers however this post is really a good piece of writing, keep it up

  94. You’re so awesome! I don’t believe I have read a single thing like that before. So great to find someone with some original thoughts on this topic. Really.. thank you for starting this up. This website is something that is needed on the internet, someone with a little originality!

  95. I have been browsing on-line greater than 3 hours as of late, but I never discovered any fascinating article like yours. It is lovely worth sufficient for me. Personally, if all site owners and bloggers made just right content as you did, the net will likely be a lot more helpful than ever before. “When you are content to be simply yourself and don’t compare or compete, everybody will respect you.” by Lao Tzu.

  96. Excellent post. I was checking continuously this weblog and I’m impressed! Very useful info particularly the final section 🙂 I care for such information much. I used to be seeking this certain info for a very lengthy time. Thanks and best of luck.

  97. Jameshek says:

    https://over-the-counter-drug.com/# best over the counter acne treatment

  98. I got what you mean , thankyou for putting up.Woh I am glad to find this website through google.

  99. My brother suggested I would possibly like this website. He was entirely right. This submit actually made my day. You cann’t consider just how much time I had spent for this information! Thank you!

  100. Wonderful beat ! I would like to apprentice even as you amend your web site, how could i subscribe for a weblog site? The account aided me a applicable deal. I have been tiny bit acquainted of this your broadcast provided vivid transparent concept

  101. ThomasJah says:

    https://stromectol.science/# order minocycline 50 mg online

  102. I like what you guys are up also. Such clever work and reporting! Carry on the excellent works guys I’ve incorporated you guys to my blogroll. I think it will improve the value of my web site :).

  103. Hello! I just would like to give a huge thumbs up for the great info you have here on this post. I will be coming back to your blog for more soon.

  104. I have been surfing on-line greater than three hours lately, yet I never found any attention-grabbing article like yours. It is pretty worth enough for me. Personally, if all website owners and bloggers made just right content material as you did, the internet can be much more useful than ever before.

  105. I and also my guys have already been analyzing the good secrets and techniques on your web site and then immediately I got a terrible suspicion I never expressed respect to the web site owner for those strategies. The young boys were definitely totally happy to see them and have extremely been enjoying them. Appreciate your simply being considerably kind as well as for choosing variety of fantastic subject areas most people are really wanting to know about. My sincere regret for not expressing gratitude to earlier.

Leave a Reply

Your email address will not be published.