The medical, public health and economic consequences of bad diagnostics

10 Aug 2011

Madhukar Pai, David W. Dowdy and Karen R. Steingart

Source: TropIKA.net

Comments

TB serodiagnostics as a case study.

This guest article is from Madhukar Pai, David W. Dowdy and Karen R Steingart, TB researchers at McGill University, John Hopkins Bloomberg School of Public Health, and University of Washington School of Public Health. Their studies published this week formed the basis for a historic first negative policy recommendation in the field of TB, from the World Health Organization, against inaccurate antibody-based serological tests for active tuberculosis.

On 20th July 2011, the World Health Organization (WHO) published a policy recommendation against the use of currently available commercial blood (serological) antibody-detection tests to diagnose active tuberculosis (TB). WHO has urged countries to ban the inaccurate and unapproved blood tests and instead rely on accurate microbiological or molecular tests, as recommended by WHO.

While serological tests work well for several infectious diseases (e.g. HIV, hepatitis B), serological tests for TB are inaccurate, inconsistent, and ultimately of no clinical value. Even though many of the existing serological TB tests are manufactured in the developed world, none are approved for use in those countries by the FDA or similar regulatory authorities.

A previous meta-analysis published in PLoS Medicine in 2007, commissioned by TDR, demonstrated their lack of accuracy, and a large-scale study of 19 commercial rapid serological TB tests by TDR the following year confirmed those findings – none of the serological tests had acceptable performance for TB diagnosis.

Unfortunately, after these studies were published, the evidence was never translated into policy. While positive results often generate publicity and excitement, negative findings rarely evoke action. Since no international guideline had ever recommended the use of serological TB tests, it was unclear how to recommend against them.

While most regulatory and guideline bodies have mechanisms in place to withdraw or ban irrational/dangerous drugs and vaccines (products that are physically put into people’s bodies), there is little awareness about the consequences of bad diagnostics. In fact, there is little published data on the human and economic impact of inaccurate and/or suboptimal diagnostics, although common sense tells us that inaccurate tests lead to misdiagnosis and mismanagement, with likely adverse consequences, both for health systems and for patients.

If any of us had a life-threatening disease like TB, the last thing we would want is for our doctor to use a test that’s more likely to give the wrong diagnosis than the right one. Misdiagnosis of TB is harmful both to individual patients and to public health. For patients, this may include loss of money, unnecessary TB therapy because of false-positive results, or morbidity and mortality because of false-negative test results. From a public health perspective, every missed TB diagnosis may result in additional TB transmission, and every false-positive result leads to wasted resources, resources that could be used to scale-up other validated, WHO-endorsed diagnostics.

One possible reason that bodies like the WHO didn’t initially take action on commercial serological TB tests was the perception that these tests were rarely used. This perception couldn’t be further from the truth. Recently, we have realized that these tests are widely abused in the private sector in countries such as India. Our research found that at least 1.5 million TB serological tests are estimated to be done in India every year at an expenditure conservatively estimated at US $15 million per year, not to mention the cost of TB drugs wasted on treating hundreds of thousands of patients with false-positive results.

When compared to the entire Indian TB control program annual budget of $65 million, this is a potentially overwhelming expense. We also found that every major private laboratory in India offers TB serological tests, mostly ELISA kits imported from countries like France, UK, USA, Germany, Canada and Australia – countries that do not approve these same tests for clinical use on their own TB patients.

While the abuse of serological tests in India has received press coverage, the problem clearly extends beyond India to most other high burden countries. Further research by our team has shown that commercial serological tests are available in at least 17 of 22 highest TB burden countries, including India, China, South Africa, Brazil, Indonesia, Thailand, Vietnam, Cambodia, Nigeria, Bangladesh, Kenya, Uganda, Afghanistan, Myanmar, Pakistan, Russian Federation and Philippines. In all these countries, the tests are used only in the private medical sector.

Regulation of diagnostics is weak in these countries, allowing for poorly performing tests to enter the market. Once on the market, financial gains by stakeholders (doctors, laboratories, diagnostic companies and distributors) keep such products profitable, providing an incentive to abuse the regulatory system, to the detriment of patients.

In 2010, recognizing the importance of tackling this widespread problem, WHO and TDR convened an Expert Group to review the evidence and formulate a policy using the GRADE approach. As part of the policy process, an updated meta-analysis was commissioned by TDR. This meta-analysis, published in this week’s PLoS Medicine, synthesized evidence from 92 studies, and re-confirmed the findings of 2007, with an even larger data set. The conclusion: commercial serological tests continue to produce inconsistent and imprecise estimates of sensitivity and specificity, and the quality of evidence remains very low.

In addition to the updated meta-analysis, the WHO Expert Group considered evidence from a cost-effectiveness study on use of serological tests in India. This study, also published in this week’s PLoS Medicine, found that as an initial test for active TB among adults in India, serology results in more human suffering, secondary infections, and false-positive diagnoses than sputum smear microscopy, while increasing per-patient costs to the Indian TB control sector. In areas where high-quality sputum microscopy is available, adding automated liquid culture (a WHO-recommended diagnostic test that generates far fewer false-positive results) is both more effective and less costly than adding serology.

Based on these two studies and expert opinion, the WHO Expert Group proposed a strong negative recommendation against serological tests in July 2010, which was subsequently endorsed by WHO’s Strategic and Technical Advisory Group for Tuberculosis (STAG-TB) in September 2010. The final WHO policy, published on 20th July 2011, concluded that “the quality of evidence for commercial serodiagnostic tests was very low, with harms/risks far outweighing any potential benefits (strong recommendation). It is therefore recommended that these tests should not be used in individuals suspected of active pulmonary or extra-pulmonary TB, irrespective of their HIV status.”

A few caveats are worth noting here. The WHO policy does not discourage research to develop improved or novel serological tests for TB based on antigen/antibody biomarkers. On the contrary, the policy clearly states that “targeted further research to identify new/alternative point-of-care tests for TB diagnosis and/or serological tests with improved accuracy is strongly encouraged.”

It is also important to note that the current WHO policy on serological tests does not include commercially available blood-based tests (interferon-gamma release assays) for latent tuberculosis infection. Separate WHO guidance is forthcoming on these blood tests for latent infection.

Lastly, the policy does not call for a ban on the platforms or technologies used for antibody or antigen detection (e.g. ELISA or rapid immunochromatography). As mentioned above, these platforms are excellent for a variety of diseases including HIV and malaria, and have revolutionized the diagnosis of these diseases in resource-limited settings.

The WHO policy against serological TB testing provides the much needed guidance at the global level. Now it is up to high-burden countries to implement this policy by tightening regulations and educating doctors, laboratories and consumers to prevent continued abuse of such diagnostics.

India has already shown leadership in this area. Immediately following the WHO policy announcement, the Indian Revised National TB Control Program (RNTCP) issued an advisory against TB serological tests. The National Laboratory Committee of the RNTCP has endorsed the WHO policy recommendations on TB serological tests and has called for dissemination of the WHO policy among all stakeholders involved in TB control in India.

The challenge now will be to reach and influence the Indian private sector, which manages nearly half of all TB cases in India. Countries such as India must find ways to incentivize the private sector to substitute serological tests with validated WHO-endorsed products, so that the economic incentives of the private sector will be aligned with the health incentives of individual TB patients.

Lastly, intensive research is urgently needed to develop accurate and reliable point-of-care tests that can replace the existing ineffective assays, offering more accurate and accessible diagnosis than what is currently possible. This will require resources, scientific collaboration, active industry engagement, and serious commitment from governments and donors.

Disclosure

None of the authors have any financial/industry conflicts to declare. All authors participated as observers (not involved in developing policy recommendations) in the WHO Expert Group meeting on TBserological tests held in July 2010. All 3 authors contributed to the cost-effectiveness analysis of serological tests in India. KRS and MP have contributed to previously published and updated meta-analyses on TB serological tests. KRS and MP are affiliated with the Stop TB Partnership’s New Diagnostics Working Group (NDWG). KRSserves as co-chair of the Evidence Synthesis& Policy subgroup of Stop TB Partnership’s New Diagnostics Working Group, while MP serves as Co-chair of the WorkingGroup. MP also serves as a consultant to the Bill & Melinda Gates Foundation (BMGF). NDWG and BMGF had no involvement in the two PLoS publications. MP serves on the editorialboards of PLoS Medicine and PLoS One.

Note: The views expressed in this article are the authors’ own and do not necessarily reflect those of NDWG or BMGF or WHO. A condensed version of this article was posted on Speaking of Medicine, the PLoS Medicine community blog on 9 August 2011.