To Test, Or Not To Test – The COVID-19 Question

In late 2019, reports of a novel coronavirus, SARS-CoV-2, came from Wuhan, China. On the 11^th of January 2020, the first COVID-associated death was reported, and over the next few months the virus spread globally – first through Asia and the Middle East, and then onto Western Europe. The current coronavirus outbreak has now reached every corner of the globe, resulting in 114 million cases and 2.5 million deaths worldwide as of March 2^nd, 2021.

Measures taken against the virus have primarily revolved around short-term strategies aimed at reducing transmission. Many governing bodies have established coronavirus regulations, focusing on hand washing, use of face masks, and social distancing. Some governments have also implemented periodic regional ‘lockdowns’, in which citizens are prevented from socialising, visiting hospitality venues, or, in some cases, leaving their homes.

The WHO has also recommended continuous tracking of the virus through track-and-trace systems, which enable governments to employ more adaptive stringency measures. The central pillar of track-and-trace systems is mass testing – testing a population allows viral transmission to be tracked, and infected persons isolated, preventing further spread of the disease.

Data from the Organisation for Economic Cooperation and Development (OECD) has shown that reduced infection rates are observed in countries with a higher number of tests per million inhabitants. Testing not only contributes to virus containment, but also provides essential information on viral mutations, population immunity, and patterns of asymptomatic transmission. In high-income countries (HICs) such as Germany, Iceland and South Korea, track-and-trace systems have been readily employed and widely hailed as successful.

A developing dichotomy
While data on infections and deaths is readily available for HICs, the picture for LMICs is far from complete. LMICs make up around 50% of the global population, but, as of May 2020, accounted for only 2% of the global coronavirus death toll. Just five HICs (the US, UK, Italy, Spain, and France) accounted for 70% of all global deaths.

While data on virus infections and deaths is readily available for HICs, the picture for LMICs is far from complete.

The dichotomy between LMICs and HICs death tolls is surprising. A multitude of factors such as household size, sanitation access, increased prevalence of vulnerable groups, and the difficulty of enforcing social distancing policies in dense, informal living situations such refugee camps or favelas, should all push the LMICs death toll higher than those observed in HICs.

The current coronavirus statistics are, however, likely a huge underrepresentation of the true disaster occurring in LMICs. Better quality data, as well as information on the implementation and knock-on impacts of stringency methods is urgently required to capture the severity of the ongoing crisis in LMICs.

The pitfalls of CFR
In their recent study, Dr Ibarra-Armenta and Dr Alarcon-Osuna sought to assess the true case fatality rates (CFR) across LMICs and HICs in order to shed light on socioeconomic drivers behind the COVID-19 death toll. The CFR is defined as the total number of deaths divided by the total number of infections in the time period specified (the beginning of the pandemic to May 7^th, 2020). There are some acknowledged limitations to the use of CFR, however.

One limitation is that different countries have different criteria for test administration. For example, in the UK, tests are administered only to those with suspected exposure, who display COVID symptoms, or who are hospitalised for COVID-associated complications. By contrast, in Iceland, mass testing of the whole population has been employed. The total number of documented coronavirus cases (the denominator in the CFR calculation) will therefore be an underestimation. There is no current agreement on the size of this underestimation – it has been postulated to be anywhere from 5% to 80%.

Countries also differ in the way they record deaths. In the UK, until recently, the Department of Health counted all deaths in which patients had tested positive, regardless of what actually killed them. Timing is also paramount; once an individual is infected, it can take up to two weeks for symptoms to appear. When patients are hospitalised and receive intensive medical care, it could take weeks or months for fatality to occur, which can warp daily data.

Dr Ibarra-Armenta and Dr Alarcon-Osuna aimed to use statistical models to link changes in the dependent variable (CFR) to a set of independent variables. These included health indicators (obesity, diabetes, elderly population proportion and co-morbidities), tests per million inhabitants, and the Stringency Index (SI). The SI is a composite measure based on indicators that record information on containment and closure policies, such as lockdowns.

Model mania
This study utilised three statistical models (the Ordinal Probit model, the Cross-Section model, and the Panel Fixed Effects model) to assess data on coronavirus cases, deaths, testing systems, health indicators (assessed as regional expenditure on healthcare) and SI across 71 countries, from the beginning of the pandemic up to 7^th May 2020. Data was sourced from the ‘Our World In Data’, WHO and OECD databases.

The Ordinal Probit model was estimated using Stata software. An Ordinal Probit is a type of regression analysis, used to reveal casual relationships between dependent and independent variables. In this model, the dependent variable is ordinal, that is, it ranks the countries according to their CFR. In this way, the fatality rate rank was then linked back to the independent variables described above, and inferences were made about the impact of testing, health and economic indicators, and SI policies on the ongoing COVID outbreak.

The Cross-Section model was estimated using Least Ordinary Squares (LOS) linear regression analysis in Stata. The Cross-Section model builds on the Probit model but uses both cases and deaths per million inhabitants as the dependent variable, instead of CFR. The aim of the Cross-Sectional model is to support results shown by the Probit model, by presenting a correlation between the number of cases and deaths and the independent variables.

The Panel Fixed Effects estimations were used to test the robustness of results from the Probit and Cross-Sectional models. Panel Data models combine Cross-Sectional analysis with Time-Series analysis; Time-Series analysis involves taking a sequence of data points at successive, equally spaced points in time. As we have seen, the aim of the Cross-Sectional model was to link average country CFR to annual changes in the independent variables using one complete, static data picture. The Panel model uses daily changes for both dependent and independent variables, and therefore allows the viewer to see dynamic, day-to-day links. Combined, the three models should contribute to a more well-rounded understanding of how lifestyle, economic and stringency factors contribute to coronavirus fatalities.

Correlation does not equal causation
The Ordinal Probit model and the Panel Fixed Effects Model displayed a negative correlation between tests per million inhabitants and CFR, indicating countries that test a higher proportion of their population have fewer fatalities. The Ordinal Probit and Cross-Section models showed a positive correlation between higher expenditure on healthcare and CFR, indicating that countries that spend more money on testing and hospital care identified more positive cases and more deaths. For all models, the SI was not significant in relation to CFR, but did show a negative correlation with daily deaths in the Panel model. The Cross-Sectional model also showed a positive correlation between the number of tests and the number of total cases identified.

To truly understand socioeconomic drivers behind COVID-19 CFR, further testing and a more accurate global dataset is urgently required.

These results indicate that testing is a significant factor in decreasing the COVID-associated CFR, and, in accordance with the WHO guidelines, should be deployed in the form of mass track-and-trace testing campaigns across the globe. The results also showed that countries with a higher health expenditure identify more positive cases and deaths. This relationship should not be viewed as causative, but rather as an indication that countries with more organised, established healthcare systems (such as those generally found in HICs) have the capacity to accurately record the unfolding impacts of the COVID crisis. Countries that lack such a system are unable to provide a true dataset due to a lack of funding and a subsequent lack of tests.

Lastly, the results show that the more tests deployed, the higher the number of cases identified. This relationship is not causative, but implies that the datasets are reactive, rather than a holistic picture of the total global population. While stringent containment methods, such as lockdowns, are effective for reducing daily deaths over a short interval, it is unclear if they are effective at reducing CFR over more substantial time frames. This factor has important public policy implications, as stringency measures generate huge economic losses, particularly in LMICs where the majority of the workforce is employed in manual labour activities.

A data minefield
Dr Ibarra-Armenta and Dr Alarcon-Osuna’s study demonstrates that while increasing mass testing helps decrease COVID-associated deaths, the data availability from all nations, particularly LMICs, is massively impaired, and hinders accurate analysis of CFR at the national and global level. To truly understand socioeconomic drivers behind COVID-19 CFR, further testing and a more accurate global dataset are urgently required.