Simple dipstick tests for infectious diseases are multiplying rapidly. But how good are they? Field trials are few, and results disappointing. Countries should press for more research and proper regulation.
SUMMARY: Simple, accessible and effective rapid diagnostics tests for the major diseases of developing countries could cut unnecessary treatments, reduce the development of drug resistance, and save millions of lives. But tests are under-researched and unregulated – so not a single rapid test for TB, for example, can be recommended. More field trials and proper regulation are the answer.
Pharmaceuticals are highly regulated worldwide – with new entities requiring rigorous clinical research that pushes their cost (say companies) towards US$1 billion. But amazingly, in many countries anyone can develop a ‘diagnostic’, make a claim to its effectiveness, and go ahead and sell it.
At the same time, biotechnology is making testing easier and more accessible, using “dipsticks” that change colour when they detect pathogens in whole blood. If they work, they can ensure appropriate treatment even in remote areas without electricity. So the numbers of rapid diagnostic tests are multiplying rapidly.
In principle, good and simple tests can save lives, pharmaceuticals – and health budgets.
For example, a study funded by the Bill and Melinda Gates Foundation showed that diagnostics of 95% sensitivity and specificity, backed up by treatment, could save 1.8 million lives a year from malaria, and prevent 400 million unnecessary treatments (see Table for more examples).
Early diagnosis and treatment can also reduce the risk of a patient developing long-term complications – and for diseases such as tuberculosis, sexually transmitted infections (STIs) and HIV, they also reduce further transmission of disease to other members of the community. And over the long term diagnostics and the consequent reduction in misuse of important pharmaceuticals can reduce the development of drug resistance, like that of the malaria parasite to over-used chloroquine.
However existing rapid diagnostic tests have a limited shelf-life six months or less in tropical climates, cost from $0.50 – $1, are largely unregulated, the published literature gives little guidance to their value, and many are failing in the field. So better tests need to be developed, and they all need independent testing where they will be used.
So Rosanna Peeling, head of diagnostics research and development at the Special Programme for Research and Training in Tropical Diseases (TDR) tells RealHealthNews.
(May 07)
RHN: In much of the developing world, are diseases mostly diagnosed by their symptoms?
RP: Yes, by clinical presentation… There are some laboratory tests. But because in the developing world there aren’t very many laboratory facilities, the WHO recommends the integrated management of childhood illness (IMCI), where health workers look for certain signs and symptoms in the patients, and then use broad spectrum antibiotics to cover major causes of those symptoms.
For STIs the WHO advocates ‘syndromic management’ to compensate for the lack of access to laboratory testing facilities.
So, a lot of the diagnostics that had been developed up to the 1990s are mostly for developed world use…
RHN: …except for microscope work to detect parasites, for example. But of course not every village has a microscope! When is it medically dangerous not to know exactly what someone has got?
RP: There are two scenarios. One would be where infections cause no symptoms at all and yet the consequence are very serious. So for example, most people who acquire syphilis may have very transient or no symptoms.
Women who became infected with syphilis during pregnancy, or before they became pregnant, would have no idea that they had the disease.
Then what happens is that the infection is passed on to the baby, and the woman either miscarries, or that the baby is stillborn, or born with congenital syphilis – which has very serious consequences, lifelong.
The magnitude of the problem is that, for example, in sub-Saharan Africa, half a million babies die a year because of congenital syphilis.
This is simply because we fail to screen the infection in women in pregnancy.
The other scenario is that you or your child may appear ill, but your clinical presentation is very non-specific. An example of this would be malaria, or children presenting with fever.
So, in Africa, a child presenting with fever in a malaria-endemic area, would be treated for malaria, and if the child also has shortness of breath, you would treat for both malaria and for pneumonia.
But in Asia, a child presenting with fever could have dengue fever, they could have malaria, could have several other causes of fever…
RHN: And in each case you’d do something different to save the baby.
RP: Yes, because the broad use of antibiotics is not a good thing, as it can lead to drug resistance.
Also, in a malaria-endemic region, malaria drugs are becoming very expensive, because the parasite has developed resistance to all the cheap drugs. And so we can no longer afford presumptive treatment of every child presenting with fever.
So, now we have to really target the treatment, but how do you do that? What indications can you use? In areas where there is no microscopy, there is no alternative. But you can lose a child in a few days from malaria.
RHN: But with modern technology, particularly with genomics, it must be becoming easier and cheaper to do these diagnoses.
RP: For malaria alone, there are 40 companies making dipstick-type tests, and in principle these tests work very well. All you have to do is to put a drop of whole blood onto the dipstick, and then add a drop of reagent, and you watch this blood migrate across the stick.
And when you see two lines, that means the test is positive. When there’s only one line it means that the test is negative.
RHN: It’s a simple concept.
RP: 80% of rapid tests on the market, for whatever disease, are of this type.
RHN: You say 40 companies are making such things for malaria, but there’s no regulation. So you decided to test them?
RP: Yes. We decided to develop an evaluation scheme to systematically evaluate the performance of these tests, and not only the sensitivity and specificity, but also the user friendliness.
We asked: are the tests simple to perform? Are the instructions clear? Is it easy to read the line?
And one of the key things is that if this test is to be performed in a peripheral clinic where there’s one health worker who’s looking after many patients, you may not be able to wait the 20 minutes for the lines to develop.
Sometimes you may be busy with another patient or with other things, and then you have to come back an hour later, or two hours later. Would the test still give you the same result? With some of the tests, after one hour or two hours all tests would come back positive – because they absorb moisture from the air and then all the lines would show up.
So, we have to do these sorts of practical evaluation.
RHN: This is a very interesting situation in medicine for developing countries – it’s a reversal of the usual case. You’re not short of products. The products are all over the place. But you don’t know how good they are!
RP: Yes, at least for some diseases. For other diseases like sleeping sickness, there are hardly any products out there on the market.
And also the current test for sleeping sickness require a spinal tap to stage the disease to give the correct treatment, because the treatment for acute infection and the treatment for the disease that has gone into the neural system are different – and both of them are toxic so you need to be sure that you’re giving the right drug to the patient.
We really need a type of diagnostic that uses a less invasive specimen to replace the need to do a spinal tap.
It’s similar for visceral leishmaniasis, where you have to do a splenic aspirate, or a lymph node biopsy. Presently, the gold standard is the splenic aspirate – but it carries with it a mortality of 1:2 000.
The governments of India, Bangladesh and Nepal declared that they’d like to eliminate visceral leishmaniasis from the Indian subcontinent, at the 2005 World Health Assembly. So, if you think in terms of elimination, and you’re going to use diagnostics widely to find cases…
RHN: …A lot of people would die just from the diagnosis.
RP: Exactly. So, there is the impetus for us to really find a non-invasive way, or a less invasive way of diagnosing visceral leishmaniasis.
For different diseases we’re at different stages. Our three areas of focus are test development, test evaluation, and then application – doing demonstration projects on the feasibility and utility of these tests.
For African sleeping sickness, leishmaniasis and tuberculosis we’re mainly working upstream in terms of facilitating better tests to be developed, and evaluating whatever tests there are on the market.
But for schistosomiasis and dengue, our work is mainly evaluating existing tests with a little bit of redevelopment if certain tests need improvement.
For malaria and STDs we’re way downstream in terms of doing demonstration projects on cost-effective strategies to increase access to effective diagnostics.
For syphilis, we’ve actually included seven tests with acceptable performance into the WHO’s bulk procurement scheme so that member states can have access to quality-assured tests at negotiated prices.
RHN: In your trials, did any of the syphilis tests come out badly?
RP: Only one, and, and it didn’t do that badly. And I think there is a reason for that. Many of these tests were developed for blood-borne infections. I think manufacturers who want to get that blood screening market try harder.
RHN: You said there were about 40 companies making tests for malaria – what spectrum of quality did they have?
RP: Some of the publications show sensitivity as low as 30-40% and some are in the 90% region, so there’s a whole range of them.
But the problem with relying on published data is that a lot of the studies are done in not very well-defined populations, so that you don’t really know whether the test was evaluated in the populations of intended use.
RHN: Are they producer-sponsored tests we’re talking about here?
RP: Yes, normally.
RHN: So there’s bound to be some bias!
RP: Well, maybe it’s unfair to always say that there would be bound to bias! I always say they’re more prone to bias. Sometimes it’s not so much the manufacturer’s fault as that they are trying to market their test for every kind of population.
We try to define in what setting these diagnostic tests should be used, and in what setting shouldn’t they be used. Whereas for companies, all they want is to push their test. So sometimes it leads to a lot of very misleading data.
On top of that, companies have varying budgets for doing these sorts of evaluations. So, we did a survey a couple of years back to ask companies how much money they actually spent on evaluations, and the range is from some US$2 000 to US$2 million.
The company that spent US$2 000 actually evaluated their test on 15 patients… and the test is sold. But that wasn’t a malaria test.
RP: Recently a company sent a syphilis test to me for evaluation, and the claim in the product insert is 100% sensitivity, and 100% specificity. I said, wow, this is great, this is exactly what we need! So, I phoned them and I asked how the evaluation was done.
They said, well, we contracted it out so we don’t really know. And I said, how many patients? Oh, over 100 patients. I said, how many were positive? Three!
So this is the kind of data you find in the literature. So another thing that we ended up doing this past year has been to write and develop recommendations on how to design and conduct diagnostic evaluations, because there are just no standards out there.
We’ve done that for malaria, and we’ve done that for sexually transmitted infections, for syphilis, gonorrhoea, and chlamydia.
RHN: Among all the diseases, which one came up worst in, in the evaluation of all the tests?
RP: Tuberculosis.
RHN: And how bad was that?
RP: Some tests had sensitivities of 1%, if that’s possible [laughs].
RHN: You mean in just 1% of the positive cases they actually gave a positive…?
RP: Yes.
RHN: Wow. And what was the best?
RP: Out of that evaluation, the best sensitivity was around 60-70%. Not good enough.
RHN: How many tests did you look at in TB?
RP: Nineteen, and none could be recommended.
RHN: What’s making the tests so bad?
RP: Well I think it’s because the test developers don’t evaluate them in the right population – where they don’t have good access to good culture facilities, microscopy etc… And also because of the lack of regulation – they develop something and they put it on the market.
RHN: And you have to think of other practical criteria too.
RP: Yes, for example the tests should have long stability – because when these tests get shipped to a country, they could sit in customs for six months, and then, after they clear customs, and then it still has to be sent out to the regions.
So if you have a test with a year’s stability, or six months, it’s certainly not enough. A year, barely, because by the time it gets to the regions it may only have three months’ viability. And then for some countries, of course, the bottom line is price.
Countries also always ask for technical assistance when they purchase tests, because we give them training modules on how to train people to use the test safely.
When you roll out these tests to remote areas where they’ve never taken a sample, they’re afraid, especially in Africa, of HIV transmission, etc. – so doing it safely at a health post, how do you dispose of the test, how do you dispose of the blood, these are issues.
RHN: Well, these are important questions…
RP: Yes, very, and they do affect the practical choice.
RHN: Broadly, many of the current rapid tests have serious weaknesses – though some, for example the antibody tests for syphilis, seem stronger. In developing improved tests, what would be the most important challenge to tackle?
RP: Right now for us it would be to increase sensitivity – while keeping the test accessible. Remember there are wonderful tests that you can do for TB, dengue, STIs and so on, in sophisticated laboratories; but they require expensive machinery, electricity, and highly trained staff.
RHN: So we really need to increase the volume of R&D on developing new tests that combines good performance with ease of use in primary health care settings – and then conduct field trials in developing country circumstances.
RP: Oh yes. Getting the right diagnostic tool is only half the battle. The other half is getting the testing done properly in the field and ensuring treatment.
RHN: To what degree do you think you need to scale up field testing and comparison of tests? By two, by ten, or what would you say?
RP: I would say by ten! At TDR we are now dealing with diagnostics for nine diseases with five professional staff. And TB and malaria are really huge diseases.
RHN: So you could do with other partners.
RP: Yes, we already have about 30 partners in developing countries, like the Tanzania National Institute for Medical Research (NIMR), the Kenyan Medical Research Institute (KEMRI). We also collaborate with Foundation for Innovative New Diagnostics (FIND), the Program for Appropriate Technology for Health (PATH) and a number of tropical disease research institutes in the developed world.
RHN: But you need to scale up in terms of numbers of people and also budget?
RP: Yes. But the Gates Foundation has given a lot of money for TB diagnostics, and are providing some funding for our work on STIs and malaria; and FIND has some funding for African trypanosomiasis – so donors are beginning to recognize the need.
RHN: In countries, in terms of health budget, would a diagnostic budget be a lot cheaper than treatment, in general?
RP: It’s a fraction, usually.
RHN: So that makes it easier to get into policy.
RP: But for syphilis, for example, the treatment is also very cheap because penicillin is very cheap. And yet, if you don’t have the diagnostic you cannot even guide treatment. And so there the cost of the treatment is immaterial.
RHN: May I ask you about the question of regulation of diagnostics? Is there any move towards tightening up regulation?
RP: Yes, part of our plan for next year and beyond is to try and help countries with developing capacity for the regulation of diagnostics.
We’re hoping for two things. First is that countries themselves would devote personnel and resources towards diagnostic regulation; and second, is that they would actually do what we call regional harmonisation of the approval processes.
In other words, we hope that if an evaluation were made in Tanzania, for example, then maybe all of sub-Saharan Africa countries could accept this data, instead of insisting the manufacturer go and do a study in their country to obtain country-specific data before approval.
So most people underestimate the power of diagnostics. So if there’s anything to be done about making the role of diagnostics better known, and how the research on diagnostics can lead to saving lives, it should be done.
But there’s no media pressure. When you see the headlines, it’s never about diagnostics. Diagnostics doesn’t cure anyone. Drugs, vaccines, are the end product of what people want. Diagnostics is neglected because its role is not really clear.
RHN: And but who needs to know, to hear this story?
RP: I would say the disease control programmes certainly need to pay more attention. Countries need to pay more attention in terms of regulation.
RHN: They need to be aware, actually, of how bad some of the tests they’re buying are.
RP: Yes, exactly.
Read on
Lives or DALYs to be saved by diagnostics (per year) – some examples
Malaria
A rapid test with 95% sensitivity and specificity and minimal laboratory requirements could save 1.8 million lives and prevent 396 million unnecessary treatments.
TB
A rapid test requiring no laboratory infrastructure, with 85% sensitivity for spear-positive and smear-negative cases, and 97% specificity, could save 400 000 lives.
Acute lower respiratory infections (ALRI)
A rapid test for bacterial ALRI with 95% sensitivity and 85% specificity with greater treatment access and minimal laboratory requirements could save over 400 000 lives.
HIV/AIDS
A rapid test with 90% sensitivity and specificity and minimal laboratory requirements could save 180 000 DALYS if 5% of the population had access to ARTs, rising to 2.5 million DALYS if 100% of the population had ART access.
Syphilis
Screening of antenatal women with a test that is at least 86% sensitive, and 72% specific, and minimal laboratory infrastructure, and has either everyone returning for their results or everyone being treated could save over 138 000 lives and prevent 140 000 stillbirths. If no laboratories were required the test and treatment could save over 201 000 lives and prevent 215 000 stillbirths.
[Extracted from Mickey Urdea et al., Requirements for high impact diagnostics in the developing world, Nature S1, 73-79 (2006), “Determining the Global Health Impact of Improved Diagnostics for the Developing World”]
——————-
Rosanna Peeling: I remember being told, and I wouldn’t name the country, there was an outbreak of dengue and there was no time for evaluating which test was good, so they just bought some tests at the recommendation of someone inside the ministry.
And then when they found that they weren’t picking up many cases, they sent these tests to a university lab that is famous for doing dengue research, and they found that the tests they ordered were really poor.
And so it was only then that the ministry realised that they should have put into place some evaluation mechanism.
RHN: But the tests were marketed as 100%, presumably!
RP: Yes, exactly. The ministry just looked at all the product inserts and said, OK, we’ll take this one.