Let's make health systems work

The Alliance for Health Systems and Policy Research will expand health systems science globally, and put it to work for policy-makers. Manager Sara Bennett tells RealHealthNews how.

SUMMARY
The communication of research to policy is a little-studied art, and needs the synthesis of results by knowledge brokers, and testing in focused, country-level contexts, argues Sara Bennett of the AHPSR. Health systems research itself is also deeply contextual, and much more investment in multicountry studies is needed if we are to provide good evidence to policy-makers, she says.

(May 07)

RHN: Worldwide, there is an increasing interest among donors on communicating research to policy-makers. Do you think researchers should be encouraged to take this into their own hands, and promote their results?

SB: I think the focus on disseminating research findings and communicating them to policy-makers is important, but I think there’s a problem if you say each individual research project should be communicating to policy-makers….

We need to look more at knowledge brokers who have a role in synthesising the research evidence available, from within the country as well as from other countries, reflecting and adapting it to local country contexts.

RHN: One of your goals within the Alliance is to synthesise results in this way. Is this what appealed to you about the organization, when you decided to join as Manager?

SB: Absolutely, in that I’ve been involved in health systems research and the application of research to policy, since the beginning of my career; at the same time I felt, and other stakeholders in the Alliance felt, that for this agenda to be really successful it has to be owned much more by the users of health research than it has been to date.

I think this is quite a common problem with health research initiatives, that the researchers have a very strong stake, but the people who need to use the research have a much weaker stake.

So one of the things that we’ve been trying to do is to shift the Alliance a little bit from research and knowledge generation to appreciating that while that is very important, it is only one part of the process. You also need to look at how research evidence is synthesised, and how policy-makers and civil society organizations can access research evidence and use it in their day-to-day work.

RHN: In journalism, we’ve found a similar problem with the media, particularly in developing countries, in that we are getting an increasing number of science writers and broadcasters, but they seem to know very little about approaching the policy world.

So they are in good contact with their scientists, and with scientific results, and can explain them clearly, but they have little experience at all of dealing with the ministries. So I think there is a need for developing a culture of reporting that crosses that division as well.

SB: I think that’s quite relevant, because there is also a somewhat simplistic model of researchers on the one side, and policy-makers on the other, and then research being communicated to policy-makers who then make the decisions.

But in fact there is a whole set of different actors involved in policy networks, media being a very very critical part of that.

We were in a meeting last week, and the deputy director of health services in Ghana was saying that if the media cover a story, and the Minister sits up and listens, that really brings things to the policy table.

So I think it’s important to understand all of these different actors and organizations and how they inter-relate.

RHN: Covering health policy and systems research globally is a large task. The goals of the Alliance as stated in your documents are extraordinary, but you have only 3-4 staff. Do you have the resources to do the job?

SB: Coming at the question in another direction, when we were working on our strategic plan there was some move to shrink our objectives to match the resources currently available; and a number of us resisted that. Because all our objectives are important ones – goals that the world should be trying to meet.

I guess a frank answer to you question would be no, at this point in time we don’t have the necessary resources or staff to meet those objectives fully; but we need to put them out there, and demonstrate that they are important.

RHN: You have a grant of £5 million (US$9.9 million) from the UK’s Department for International Development (DFID) – is that your main funding?

SB: DFID is the most significant single funder. Our total income is about US$3.2 million a year. This year [2007] we are projected to spend about US$4 million, because we’ve had a slack period and we have some money in the bank.

RHN: And how are you dividing that cake, roughly?

SB: We have three objectives: knowledge generation, evidence-to-policy work and capacity development. We were striving to get a fairly equal balance between the three. In practice, for this biennium, capacity development is going to be rather less than the other two.

In knowledge generation we are focusing first on identifying the agenda, the priority research questions in our field, and second on synthesising the knowledge that already exists, pushing forward on systematic reviews.

And on evidence-to-policy work, it has become clearer and clearer to me that the evidence base for this type of work is quite limited. A few countries like Canada and the UK stand out as having good mechanisms and systems for getting evidence into the policy-making environment, but there has been very little experimentation with those kinds of mechanisms and systems in low- and middle-income country contexts. So it’s a little bit moving out into the unknown.

So in this area what we are trying to do is to focus on a handful of countries, and support them on implementing some of these mechanisms, and then to evaluate what seems to work and what doesn’t.

RHN: These are mechanisms for getting research or knowledge into the policy environment?

SB: Absolutely.

RHN: We are all learning about this as we go along, but what is your current view as to what look like the beginnings of good strategies?

SB: I think firstly that the evidence so far suggests that this kind of evidence-to-policy transfer is more likely to happen where the policy networks are quite rich – in the sense that these actors, researchers, civil society organizations, media and policy-makers do already interact quite frequently.

Much of the activity of WHO’s EVIPNet (the Evidence-Informed Policy Networks) has been trying to stimulate those kind of network, and I think that that is very important; but I also think that it’s likely to work better when you are doing it around specific policy issues.

So what we are trying to do at the Alliance is to work with policy-makers to identify priority policy issues that are coming up on their agenda within the next couple of years, and to then use that as a vehicle to build networks around.

For example we are commissioning policy briefs on specific issues that might reflect the systematic reviews available; to conduct ‘safe harbour fora’ or ‘deliberative fora’ where you bring all of those members of the policy network together under Chatham House type rules, with open dialogue but all off-the-record.

We’d like to open up a discussion between the researchers and the policy-makers, civil society organizations, and as far as possible the media, about what the research evidence is and how that relates to policy.

RHN: Are your priority countries the same as EVIPNet’s?

SB: EVIPNet is working in a lot of different countries; we’re trying target our resources a bit more, and not just our financial ones but also our technical support.

So we are focusing on Vietnam, Kyrgyzstan, and the Regional East African Community Health (REACH) policy initiative countries – Kenya, Uganda and Tanzania. We are providing grants to all three of those initiatives.

Also, given our nature we are focusing more on health systems issues than EVIPNet, which is wider.

RHN: And the issues that you are looking at in these countries – has the initiative come from them?

SB: In all cases. Yes. First the interest in looking at research-to-policy mechanisms has always come from the countries and typically has some degree of policy-maker buy-in.

But to be quite frank, I think that is one of the biggest struggles at this point - getting real policy-maker buy-in. Some policy-makers are already interested in research, but others – who may be more influential – are harder to get to the table.

This is one of the reasons we decided to focus on specific policy issues that we know will soon be on policy makers’ agendas. Because when they see how this can be instrumental to the work they are trying to do, then it will be easier to get them to the table.

RHN: As for your policy briefs – will these be based on systematic reviews?

SB: To be honest I think it is a little bit undefined right now, for a number of reasons.

Firstly, if you look at the methodology of Cochrane-style reviews, which focus on the effects of different interventions, then in the health policy and systems research field there are relatively few evaluations of policies or interventions that give you enough data to do a really good systematic review.

So for example colleagues at the London School of Hygiene and Tropical Medicine have been working recently on a Gates’-funded review of different health financing mechanisms, but for things like social health insurance there are simply no studies that meet the type of standards typically used for Cochrane reviews.

RHN: Is that so! And yet there are demonstration projects in social health insurance all over the place. But no-one has studied them properly?

SB: Well Cochrane identifies certain types of study design as being best suited to answer questions about effectiveness. For example a randomized controlled trial would be one of these; but how on Earth could you do a randomized control trial of a social health insurance intervention?

So while some evidence can come from those kinds of studies, our sense is that often they are going to be insufficient for policy-maker needs.

After all, policy-makers are concerned not just about whether social health insurance works or not - they want to know how best to implement it. They need to know how to communicate it to the population, which part of the population to cover first, and so on.

So there are a lot of ‘how’ and process questions that policy-makers are typically interested in, that systematic reviews of effects don’t capture.

And once you begin to move away from that core, Cochrane-type of systematic review, the methodologies are much less well-developed and much more contested.

RHN: Of course in this area of social and public health you are moving much closer to politics, aren’t you, which is where a lot of these policy-makers are sitting, and the political and cultural influences in countries become very important.

SB: Absolutely, yes. And that’s the other big question around these reviews – if contracting for health care services is shown to be effective in Mexico, Brazil and Argentina, what can we draw from that experience for Uganda, Tanzania and Rwanda, where conditions are very different?

So one of the real problems facing health systems research is how we generalise from a study in one country to another.

My personal feeling is that because health policy and systems research has been so underfunded, we’ve got a scattering of individual country studies, often using slightly different research approaches, methods, study designs, that haven’t looked at contextual issues systematically in the same way across countries.

So I think we need more multicountry studies to try to address some of these questions, in a way that really looks at how context affects what does and doesn’t work.

RHN: Let me ask you about another broad issue. There is a tone that one hears in a lot of the literature and argument about the need for health research in driving health policy, that actually sounds more like ‘selling science’ than actually helping policy-makers. And I can imagine particularly policy-makers looking at that from their political perspective and saying ‘this is just another interest group facing us’.

So what evidence can we give them that research really is effective in helping them make complex policy decisions?

SB: [laughs] A very good question! I’m not sure that there is huge empirical evidence that supports that. You’ve got the recent series by Julio Frenk in The Lancet discussing the success of the evidence-informed policy process in Mexico; but you can’t have another Mexico facing the same policy issues and making decisions without any of the evidence that Mexico used! So it’s difficult to be clear about the difference that research made.

I think most people, if you put to them the question ‘would you prefer to have a policy that was informed by the evidence that was available, or a policy that was un-informed?’, would typically go for the policy informed by evidence.

It’s more a question firstly of the nature of the evidence that’s available; then whether it’s in a format that can be accessed easily; and then whether the knowledge base really exists in a form that would be useful to making a policy decision.

Also I think it’s a matter of recognizing that evidence is just one among many factors that are going to influence policy, and being up-front about that. We need to admit that that’s absolutely fine – that sometimes values or political circumstances will outweigh what the evidence has to say, but let’s at least be clear and open about the evidence to the extent that we can.

RHN: Let me just put to you a point that was made to me by an observer at the WHO IDEAHealth meeting on evidence for policy in Khon Kaen in Thailand last December – that the policy makers present had said that what they really needed was not so much evidence, as simple financial and human resources!

SB: We were involved in preparing some of the evidence for that meeting, but I would share some of the criticisms. One of the challenges at Khon Kaen was that there weren’t clear policy questions driving the agenda. There was a lack of focus.

There had been a series of e-mail consultations with EVIPNet groups in different countries, asking what policy-makers would like discussed at Khon Kaen, but the results had been complete topics like ‘maternal and child health’. But that’s not a policy question, it’s a whole set of issues.

And one of the things that came out of Khon Kaen, I thought quite clearly, was that that kind of discussion forum would be much much more effective at country level: they wanted country dialogues. I think then you could make it much more specific, so it’s not just about ‘maternal and child health’ but, say, ‘how to extend the integrated management of childhood illness (IMCI)’, for example. Or closer to the areas I know, ‘how do you retain health workers in rural areas?’ – something much more specific. There are actually some good reviews done around the latter question and I think policy-makers would find it interesting to see how different countries have addressed that.

RHN: Briefly, isn’t the answer to that to make sure they have schooling for their children, pay them well and pay them regularly?

SB: That might be part of it, but whether that’s the most cost effective answer, I don’t know.

Governments have also experimented with schemes that have tried to get people to do placements in rural areas once they’ve finished school; those don’t seem to be so successful. Recruiting people from rural areas, going out of your way to give preference to candidates who’ve been brought up in remote areas – that seems to be something that looks like it might work to some degree. But the evidence on the effectiveness of all these different strategies is pretty weak.

RHN: To conclude, let’s talk about the future of the Alliance. How do you see it developing? What’s your strategic plan?

SB: There’s a couple of core issues. I continue to think that health policy and systems research, both the generation of that and its application to decision-making, continue to be very neglected. So I think that the Alliance has an important role to play in raising the profile of the field, attracting additional funding to the field, and underlining the importance of what can be done if evidence is better applied.

The second part of it is that health policy and systems research, more than any other type of research, is context specific, as we’ve discussed – and I think that for the Alliance to be successful in the long run, ie over the next ten years, we need to be looking and thinking at how we develop hubs or nodes out in developing countries. I don’t think that the Alliance could ever grow by being a large centralised research programme.

We need to get the field of health policy and systems research to grow through networks and nodes in different regions so we can be much more context specific.

RHN: Has the Alliance made any attempt to any such nodes, for policy briefs or systematic reviews, for example?

SB: Yes indeed. For example we are setting up   four   centres    for systematic review of health policy and systems research in low and middle-income countries.  They are headed by Professor Qingyue   Meng at   Shandong University, China,   Dr George Pariyo at Makerere University, Uganda, Dr Tomas Pantoja at the Pontificia Universidad Catolica de Chile, and Dr Tracey Koehlmoos at the International Centre for Diarrhoeal Disease Research, Bangladesh.

RHN: It is striking, from the figures on the Alliance website, how little is being spent on health systems research in countries, down to 0.01% of health budget in some cases….

SB: Yes, and yet there are commitments for five per cent of external support for health programmes to go into research. Donors are putting a lot of money into health systems, if you look at GAVI, the Global Fund, DFID – large amounts of money are flowing to health systems programmes - but very little serious evaluation is being done.

RHN: And why is that, would you say?

SB: It’s politically a little complicated. As programmes like the Global Fund and GAVI have tried to roll out and scale up, they’ve run into health systems constraints at every turn – so now they are beginning to look at those. But at the same time it’s a little bit awkward to acknowledge how flimsy is the evidence base that you are working from, as that undermines what you are trying to do.

One of the phrases that were used at IDEAHealth was that you have to ‘mend your boat while you sail it’. I think that’s the case with health systems research. We can’t stop people using specific innovative mechanisms such as contracting by saying ‘it hasn’t been proven’. We need to use these new and innovative ways of strengthening health systems and at the same time evaluate them seriously.

top