Originally published by The Daily Skeptic
Nature published a comprehensive study this week on cardiovascular risk including a total of over 11 million patients that has made a few headlines. The aim was to identify the cause of increased cardiac pathology. It should have been a very simple study comparing four groups:
- Not infected and never vaccinated
- Not infected and vaccinated
- Infected but not vaccinated
- Infected and vaccinated
It is hard to believe the authors did not look at these groups, but whatever was found when comparing them remains a mystery.
Instead, the following groups were compared:
- Not infected and never vaccinated data from 2017
- Not infected, including vaccinated and not vaccinated
- Infected but not vaccinated
- Infected with vaccinated people included but using modelled adjustments
When studies with huge datasets use modelling and fail to share data prior to their adjustments alarm bells should start ringing. Therefore, I took a deeper dive to see what else was questionable.
There were serious biases in the paper which need addressing but first let’s look at the critical question of myocarditis (heart inflammation).
Because of the known risk of myocarditis from vaccination it is worth looking particularly closely at the data presented on this. Oddly, for the issue of the day, the data on myocarditis was all hidden in the supplementary appendix to the paper.
The risk of myocarditis appears to be an autoimmune (the immune system attacking the heart after interaction with the spike protein) rather than direct damage by the virus/vaccine spike protein. Therefore, myocarditis could result from the virus or the vaccine. The key question that needs answering is whether vaccination protects or enhances the risk from the virus.
The authors report 370 per million risk of myocarditis after Covid infection in the unvaccinated. The contemporary control rate was 70 per million and the historic one was 40 per million. What was wrong with the contemporary controls?
They made it clear they removed those who had been vaccinated from the calculation in the Covid arm but they did not state they did this for the control arm. Did vaccination lead to a 30 per million increase in myocarditis in the control arm? Given the cohort appears to be old and we know myocarditis incidence is worse in the young a one in 30,000 incidence is significant.
What about those who were vaccinated and had Covid? Once vaccination (and modelling) were included, the rate rose to 500 per million. It is not entirely clear whether supplementary Table 22 excludes those who were not vaccinated, but given that it does not state the unvaccinated were excluded from this data it is fair to assume the 500 per million relates to the whole population.
Given the higher risk of myocarditis after vaccination one might wonder whether this study showed protection from infection due to vaccination, as this would lower risk from the virus. Hidden in the legends of the supplementary tables the authors reveal that 62% of the Covid patients had been vaccinated compared to 56% of the non-infected controls (not a great advert for vaccine effectiveness against infection).
Using the fact that 62% of the Covid cohort were vaccinated and that the unvaccinated had a rate of 370 per million, to get to an overall rate of 500 per million the vaccinated 62% must have had a rate of 580 per million (580×0.62 + 370×0.38 = 500). Therefore, in those with Covid and vaccination the rate (even after modelling) was 210 per million higher (58% higher) than the unvaccinated with Covid. (If supplementary Table 22 did exclude the unvaccinated the incidence of myocarditis after Covid would have been 35% higher in the vaccinated.) An extra 210 per million works out as an additional risk from vaccination of one in 5,000 among a relatively old population. The 35-58% higher myocarditis rates seen in the vaccinated after Covid compared to the unvaccinated was based only on diagnoses made more than 30 days after their positive Covid test. Any rise in risk in the first 30 day period was censored from the study. How high was it in the first 30 days and for the younger men? This critical question was left unanswered.
The data comprised medical records for U.S. veterans who were 90% male, three quarters white and had a mean age of 63 years.
Two control groups were selected:
- Patients who had used healthcare in 2017 and were still alive in March 2018.
- Patients who had used healthcare in 2019 and were still alive in March 2020.
These groups were compared to patients who tested positive for Covid after March 2020, with each patient being matched to one patient from each control and measuring beginning from the same day as the positive test but two years earlier for the 2018 control.
There was a significant bias between these two control groups and those who tested positive.
The Covid patients (not just those who were sick with it – all those who tested positive) were more obese, saw doctors more often, had more cancer, kidney disease, lung disease, dementia etc.
There are two ways to deal with such biases. One is to match the 150,000 Covid patients with similarly sick patients from the over five millions controls. This reduces the size of the control group but when it is already so large this should not be a concern. Instead, the authors modelled the data until the groups seemed similar. Using an algorithm they claimed the same total number of people were present in the Covid cohort, but whereas 49,407 actually had diabetes in the raw data, 11,903 (24%) no longer had diabetes according to the weighted data. Similarly, 14% were ‘cured’ of lung disease, 14% of cancer and a full 35% of the dementia patients no longer had dementia.
There was no discussion in the paper about the reasons for this unhealthy bias among the Covid patients. All positive test results were included and anyone can catch SARS-CoV-2, so the factors that increase the risk of serious disease and hospitalisation should not have biased a dataset based only on infection. Instead the authors discuss the hypothetical issue of people in the non-infected control group having Covid but not getting tested such that the damage caused by Covid could be worse than the paper reports.
It has been well established that hospital transmission dominates as a source of spread and SAGE has reported that up to 40.5% of cases could be traced back to hospital spread and a majority of hospitalised patients in June 2020 were linked to hospital spread. In Scotland, in December 2020, 60% of the acutely ill with Covid acquired the infection in hospital. Patients accessing hospital are highly likely to be less healthy than the general population. Indeed, we know that the Covid patients in the study accessed hospital more frequently than the controls. If the bias was related to hospital acquired infection then the whole study is called into question, as people who attend hospitals are more likely to be sick.
The authors picked some control conditions to attempt to show they had not introduced a bias. Given the study was about cardiovascular diseases, including those that are an immediate threat to life and those that are very common, I would have picked conditions that might kill you within a year, like lung, pancreatic or oesophageal cancer and common conditions e.g. urinary tract infections, diabetes or prostate cancer.
The authors chose three rare malignancies, all with a one-year survival rate of over 80%, and pre-invasive melanoma – why not include invasive melanoma? They then included rare conditions and odd selection of: hypertrichosis (‘werewolf syndrome’ with excessive facial hair), sickle cell trait and perforated ear drums. When the choices are so niche it begs the question of what the results would have been if more obvious choices had been selected.
The group that tested positive for Covid did badly: 13% ended up in (or began in) hospital and 4% in ICU. The mean age was 63 years which may explain part of the high percentage of sick Covid patients, but it does, again, suggest this group may have been more vulnerable than the control.
They then compared the risk of various cardiac outcomes against the controls. However, they used the same control to compare non-hospitalised patients as patients who had received ICU care. Of course, people who have needed ICU care will be more likely to have cardiovascular complications. Indeed, many of the patients may still have been in the ICU when the measuring period began 30 days after the positive test. A fair study would have only compared the ICU outcomes with the sickest people within the control group, not the average of the whole control group.
The risk to the non-hospitalised Covid patients was low for almost all the cardiovascular risk factors.
The risk to the hospitalised was higher (but remember the controls had significant biases).
Those on ICU had a much higher risk. What is not clear is how much of this is because of the virus.
It is not a surprise for people who have had an ICU stay to be unwell for some time afterwards. The risk of ICU admission for Covid was higher than for influenza, but it is important to understand how much of the cardiovascular risk resulted from the virus and how much from the stay in intensive care per se. How do these Covid ICU patients compare to other ICU patients? The paper did not say.
Similarly the paper makes no attempt to unpick how many of the Covid patients tested positive only after being admitted to hospital. If, as in other studies, a significant proportion acquired Covid in hospital, then a higher risk of being diagnosed with other conditions would be highly likely.
Having failed to examine the above two questions – how much cardiovascular disease was a confounder of hospital transmission and how much is secondary to ICU harm – the overall risk of consequent cardiovascular problems included all the above cardiovascular conditions and thereby inflated the average for the Covid population as a whole.
Nature has published this paper which presents data in an obtuse way that should never have passed peer review. The results were presented as showing how dangerous the Covid virus was for cardiovascular complications without suitable controls to enable that conclusion to be drawn. The evidence on vaccination risks was hidden and not presented in a meaningful way for different age groups. Even then, they demonstrated a significant risk of myocarditis after vaccination, particularly after then encountering the virus but this key finding was hidden in the supplementary appendix. Why?
Dr. Clare Craig is a Diagnostic Pathologist and Co-Chair of HART.
