What to Read this Month
Reliable, Accurate Information Vital to Policymaking in Africa | Njiraini Muchira, The East African
“The policies are often not implementation ready-they are big in citing evidence to justify why action should be undertaken but weak in using evidence to determine what cost-effective interventions to implement.”
The Executive Director of the African Institute for Development Policy, Eliya Zulu, makes the case for governments to invest in research that informs how to effectively implement policies, and to create training and incentives for government personnel to use evidence in decision making.
Data Roadmaps for Sustainable Development | Global Partnership for Sustainable Development Data
“Interoperability of data and systems continues to be a major barrier for the effective collection, distribution, and use of data, and a barrier for countries trying to set up more modern data ecosystems.”
The seven countries engaged in the data roadmap process have several challenges in common: filling data gaps, sharing data across government agencies and the private sector, increasing political commitment to invest in data, and developing data skills among government officials.
Scaling Social Policy: Five Lessons from Brazil | Megan Dent, Apolitical
“Because Bolsa Família already works with low-income families, the Ministry had instant access to a list of vulnerable households with children under three.”
New antipoverty and social protection programs need a lot of detailed data to identify the poor households that will be eligible for program services; in Brazil, a new early childhood development program was able to utilize already-existing data from a national cash transfer program to target beneficiaries and scale up across the country.
Public Servants and Political Bias: Evidence from the UK Civil Service and the World Bank | Stefan Dercon, VoxDex
“An experiment shows that public servants make errors when interpreting data, incorrectly concluding that it aligns with their ideological preferences.”
The article presents further evidence that even skilled professionals are subject to confirmation bias; peer review / quality assurance units, red teaming, and other strategies could help staff correctly interpret the results, and be more open to results that challenge their preconceptions.
And if you haven’t read it already, check out:
The Politics of Evidence: From Evidence-Based Policy to the Good Governance of Evidence | Justin Parkhurst, London School of Economics
“A heavy focus on individuals as the driving force to improve the use of evidence in policymaking raises two particular issues. The first has to do with the roles of researchers, who are under increasing pressure to ensure that the research evidence they produce is ‘used’ or ‘taken up’. This risks encouraging researchers to have political influence, a role that they are neither trained to do nor one that many feel they have the mandate to take on. The second problem is that such efforts can have a limited duration of impact, given that both researchers and decision makers will naturally change over time or move on from existing positions.”
Parkhurst advances a holistic ‘good governance of evidence’ approach, defined as the use of rigorous, systematic and technically valid pieces of evidence within decision making processes that are representative of, and accountable to, populations served.
What to Watch
“But just as the problem was hidden within the data, so was the solution”
Still one of our favorite examples of how administrative data can be used to diagnose and overcome challenges to effective implementation of government policies. Well worth a watch!
Do you have comments, questions, or ideas for us?
We always love hearing from you! Contact us at firstname.lastname@example.org anytime.
“Informing public policy will require both better research as well as better public sector incentives.”
Put yourself in the shoes of a government policymaker for just one day. Your responsibilities for the day are likely to include some management and administrative tasks. Perhaps you will make remarks at a public event or be called on to brief your Minister on short notice. Maybe you will end your day in meetings with external partners and constituents.
Although you may not make important policy-level decisions on a day-to-day basis, when confronted with a policy question, the extent to which you seek out evidence to inform the issue you are trying to address, will depend on your ability to access the information you need when you need it, as well as the expectations placed on you by your agency, constituents, and fellow policymakers. Even with the right intention, you may find it a challenge to systematically use evidence in your work without clear direction or guidelines from your agency on how to find, use, and communicate evidence in policymaking. You may wonder about evidence use in other agencies and direct your staff to look for examples from other contexts that could inspire a new approach in your office, and wish you had opportunities to network and learn from your peers in government.
Results for All’s work over the last year has been focused on exactly this – inspiring and accelerating progress in evidence use by bringing policymakers together to build community and learn from each other. We have highlighted what we learned about peer learning networks through our consultative and reflective research process in earlier blogs and reports, including this mapping of over 50 networks, highlights and a summary report from our peer learning workshop, and this article on the potential of networks in Africa. Here we distill these learnings into 7 insights for peer learning approaches and collaborations that aim to advance evidence use in government.
- Anchor your evidence use conversation in policymaker priorities. A sweeping conversation about systems, processes, and capabilities for evidence use can lose meaning and feel vague. Instead, identify policy or thematic priorities for your network and use them as an entry point to examine institutional capacity to produce and use evidence. Do policymakers have access to timely, quality, and relevant evidence to inform their policy priority? Are there gaps in technical know-how, analytical ability, or motivation that inhibit a policymaker’s ability to routinely find, use, and report on the evidence that informs this priority? Is there a role for knowledge brokers to play in facilitating increased interaction between evidence producers including the research community, citizens, and practitioners, and policymakers as evidence users, to better inform priorities? What types of incentives could help to facilitate the use of evidence in their specific policies? Start with the problem and help policymakers think about the systems, processes, and capabilities that can be a part of the solution – but keep the conversation rooted in addressing policymaker priorities.
- Understand policymaker attitudes toward evidence use. Don’t stop with a conversation about challenges or the tangible dimensions of an evidence use culture such as data systems, knowledge management platforms, or evaluations. Go below the surface to deeply understand why policymakers use evidence and if and how it aligns with the way evidence is used in their agency. Explore what it would take to shift underlying assumptions towards greater evidence use – is evidence-informed policymaking perceived as being too complicated, too time-consuming, or too costly? Do staff have the resources, time, and training to find, share, and use evidence? Finally, don’t shy away from engaging on the messy dimensions of the policy process, including politics and power. Take time to understand how these factors facilitate or impede evidence use in order to help policymakers create effective strategies for addressing their policy priorities.
- Pay attention to practical and experiential learning. Policymakers value opportunities to network and learn from like-minded colleagues. Specifically, we note a high level of interest in tacit “how to” problem solving learning and exchange on the political and process oriented-dimensions of using evidence, to gain experiential insights on questions such as “how did you did you develop a learning agenda in your evaluation plan” or “how did you use evidence to build buy-in for your policy?” We are inspired by the Joint Learning Network for Universal Health Coverage’s approach to drawing on member experiences to co-produce practical tools and knowledge products, and the Collaborative Africa Budget Reform Initiative’s (CABRI) problem-driven approach that facilitates experiential learning in addressing budget reform challenges. Find ways to strike a balance between expert-led presentations and interactive sessions that allow policymakers to draw on each other’s experiences to address evidence use challenges. Finally, ensure there is ample time for networking and building deep connections that can outlast an event or formal platform.
- Don’t neglect the soft skills. Beyond a technical grasp of the evidence – to find, synthesize, and use complex information in decision making – policymakers also need the skills to effectively communicate research and policy priorities internally across government and externally to their citizens. They also need communication skills to build and sustain relationships with key partners, including the research and academic communities; to engage with stakeholders in building buy-in for a policy or soliciting feedback to improve implementation; and to think strategically about longer-term evidence needs for their policy priorities. While there are many guidelines and toolkits geared towards improving how researchers communicate with policymakers, we have found fewer resources offering practical tips and guidance specifically targeted to policymakers. Engage in conversations to identify gaps in soft skills and find ways to help policymakers improve how they to talk about and act on evidence.
- Stay for a while. Relationships and networks play an important role in building a shared identity for jointly problem-solving and co-creating strategies to improve evidence use, and for encouraging the spread of ideas and practices. But it takes more than one engagement to build the kind of trust that leads to an open and honest exchange of experiences. Tacit “how to” knowledge exchange on the softer dimensions of evidence-informed policymaking, in particular, is predicated on trust and close interaction. This type of exchange does not lend itself well to a one-off workshop, so governments, development partners, and funders, please consider long-term approaches for deepening peer-to-peer learning and accelerating progress in evidence use.
- Implementation challenges keep policymakers up at night.The term policymaking can be misleading in suggesting a focus on making or creating policy, but we have heard repeatedly from policymakers that one of the biggest challenges they face is translating policy to the delivery of services for citizens. Policies that are formed at the national level without considering local-level priorities and an implementation plan are often doomed from the start. Focus on finding ways to support policymakers in taking a systematic and iterative approach to using evidence in both policy design and implementation to avoid this disconnect. A good place to start could be a diagnostic self-assessment process that allows policymakers to deeply explore and understand barriers, opportunities, and strategies for strengthening evidence use practices in policy implementation.
- Create a safe space for authority figures and doers.Make an effort to ensure all voices are heard – senior-level policymakers with the authority to approve follow-up activities as well as practitioners and mid-level managers and analysts who will take the work forward. In some contexts, you may need to convene senior-level policymakers and practitioners separately. If your network targets multiple levels of policymakers, pay attention to the dynamic in your meeting room. Have you created a safe and inclusive space that welcomes different viewpoints and perspectives on evidence use?
What’s next for Results for All? With a generous third round of funding from the William and Flora Hewlett Foundation we are exploring how to bring these insights and the resources and tools we are developing to strengthen evidence use in government, to existing global initiatives and platforms that are broadly committed to strengthening policy and practice-level decisions in government. Rather than forming a new network dedicated exclusively to evidence use, this approach enables us to build on existing efforts and to collaborate with partner initiatives and participating governments to improve the use of evidence in their mandates. We welcome your feedback on what we have learned so far and would be interested to know of any initiatives that have an appetite for engaging with us on evidence use. We’ll be sharing regular updates on our new work with partners in our blog and monthly Evidence-Informed Policymaking Reading List, which you can subscribe to by emailing email@example.com.
- Public Data Lowers Violence in Cali, Colombia
- Mapping Commuters Streamlines Transit in Nairobi, Kenya
- Using Satellite and Cell Phone Data to Eliminate Malaria in Namibia
- Climate Modeling Helps Colombian Farmers Improve Decision Making
- Better Data Improves Women’s Lives
#1. Assessing the Demand for a Global Evidence Network: Mapping Existing Initiatives and Understanding Network Lessons and Opportunities | Ari Gandolfo & Abeba Taddese, Results for All
#2. Summary Report: Using Evidence to Improve Policy Implementation| Abeba Taddese & Ari Gandolfo, Results for All
#3. Ten Reasons Not to Measure Impact – And What to Do Instead | Mary Kay Gugerty & Dean Karlan, Stanford Social Innovation Review
#4. What Kind of Evidence Influences Local Officials? A Great Example from Guatemala | Duncan Green, From Poverty to Power
#5. Evidence-Informed Policymaking and Policy Innovation in a Low-Income Country: Does Policy Network Structure Matter? | Jessica Shearer et al, Evidence and Policy
#6. Advancing Evidence-Informed Policymaking: What’s Culture Got to Do With It? | Abeba Taddese, Results for All
“To be evidence champion of the month means everything to me. It motivates me to do more and also encourage my peers in other agencies to see the value in evidence for their work. Evidence is the game changer in our efforts to become a global center of excellence in environmental protection.”
An Integrated Model for Increasing the Use of Evidence by Decision-Makers for Improved Development | Ruth Stewart et al, Development Southern Africa
Understanding Data Use: Building M&E Systems that Empower Users | Susan Stout et al, Development Gateway
Calls for Greater Role for Universities in Policy-making | Gilbert Nakweya, University World News
“Building the capacity of research in African universities would strengthen their contribution to policy-making and innovation…”
More than Optimism, Institutional Reform is Needed to Improve Evidence Use in Policy and Practice | David Christian Rose, LSE Impact Blog
We always love hearing from you! Contact us at firstname.lastname@example.org anytime.
We always love hearing from you! Contact us at email@example.com anytime.