Factors influencing estimated effectiveness of COVID-19 vaccines in non-randomised studies

Non-randomised studies assessing COVID-19 vaccine effectiveness need to consider multiple factors that may generate spurious estimates due to bias or genuinely modify effectiveness. These include pre-existing immunity, vaccination misclassification, exposure differences, testing, disease risk factor confounding, hospital admission decision, treatment use differences, and death attribution. It is useful to separate whether the impact of each factor admission decision, treatment use differences, and death attribution. Steps and measures to consider for improving vaccine effectiveness estimation include registration of studies and of analysis plans; sharing of raw data and code; background collection of reliable information; blinded assessment of outcomes, e.g. death causes; using maximal/best information in properly-matched studies, multivariable analyses, propensity analyses, and other models; performing randomised trials, whenever possible, for suitable questions, e.g. booster doses or comparative effectiveness of different vaccination strategies; living meta-analyses of vaccine effectiveness; better communication with both relative and absolute metrics of risk reduction and presentation of uncertainty; and avoidance of exaggeration in communicating results to the general public.

Automated summary

Non-randomised studies assessing COVID-19 vaccine effectiveness need to consider multiple factors that may generate spurious estimates due to bias or genuinely modify effectiveness. Steps and measures to consider for improving vaccine effectiveness estimation include registration of studies, sharing of raw data and code, blinded assessment of outcomes, and better communication with both relative and absolute metrics of risk reduction.