Evidence-Informed Policymaking Reading List | October 2019

Popular Content from 2019

Informing Civil Servants to Make Better Decisions | Daniel Rogger & Ravi Soman, VoxDev
“Informing public policy will require both better research as well as better public sector incentives.”
An interesting study that highlights the important role organizational incentives play in building a culture of evidence use in government. The authors found that when public officials are given authority over decision making, they invest in more accurate beliefs about the constituents they serve, and when they work in an organization that rewards information gathering, they do more of it.

7 Insights for Peer Learning Approaches in Evidence-Informed Policymaking | Abeba Taddese, Results for All
Over the last year, our work at Results for All has focused on exploring how to facilitate opportunities for government policymakers to share knowledge, experiences, and lessons learned in accelerating the use of evidence to inform policy. Here we distill our observations into 7 key insights and principles for peer-to-peer learning for government. Do you agree? What’s missing? Please reply to this email or comment on the blog to share your thoughts!

Uncovering the Practices of Evidence-Informed Policy-Making | Louise Shaxon, Public Management & Money
“Although the analysis is in its early stages, it does suggest that government departments and agencies concerned to implement a holistic approach to evidence-informed policy-making could consider basing their strategies on seven core practices.”
Drawing on examples from the U.S. (Results for America’s Invest in What Works Federal Index) and the UK’s Department for Environment, Food & Rural Affairs (Defra), the paper identifies a minimum set of practices to help governments take a holistic approach to evidence-informed decision making.

7 Types of Policy Makers and What they Mean for Getting your Research Used | Markus Goldstein, The World Bank
“Based on my experiences [with this] there are seven types of policy makers, and knowing your counterpart’s type might be helpful in figuring out how to pitch your discussion.”
The author suggests that by understanding the type of policymaker they are engaging with, researchers can better tailor their approach to meet policymakers where they are, and offers 7 aspects of policymakers to consider in discussions.

The Politics of Evidence: From Evidence-Based Policy to the Good Governance of Evidence | Justin Parkhurst, London School of Economics
“A heavy focus on individuals as the driving force to improve the use of evidence in policymaking raises two particular issues. The first has to do with the roles of researchers, who are under increasing pressure to ensure that the research evidence they produce is ‘used’ or ‘taken up’. This risks encouraging researchers to have political influence, a role that they are neither trained to do nor one that many feel they have the mandate to take on. The second problem is that such efforts can have a limited duration of impact, given that both researchers and decision makers will naturally change over time or move on from existing positions.”
Parkhurst advances a holistic ‘good governance of evidence’ approach, defined as the use of rigorous, systematic and technically valid pieces of evidence within decision making processes that are representative of, and accountable to, populations served.

Evidence-Informed Policymaking Reading List | September 2019

What to Read this Month

Are We Suffering from Obsessive Measurement Disorder?  | Tiina Pasanen, From Poverty to Power
“The term [obsessive measurement disorder] rejects the belief that counting everything (in government programmes and beyond) will produce better policy choices and improved management. It’s a disorder that increasingly affects the international development community.”
ODI Research Fellow Tiina Pasanen asks whether the focus on monitoring and evaluation has gone too far, and whether we can ‘count only what counts’ to leave more room for implementation and learning.

The Evidence Ecosystem in South Africa: Growing Resilience and Institutionalisation of Evidence Use | Ruth Stewart et al, Nature
“If we recognise that the use of different forms of knowledge, or ‘evidence’ for want of a better word, in decision-making in the South has existed in various forms for many decades, then we shift our thinking away from merely a technical perspective and view the approach using political, social, as well as technical lenses.”
An in-depth analysis of the evidence ecosystem in South Africa, with a focus on 5 key elements: structural foundations, organizations, investments, capacities, and innovation. The authors explore how the complexities of these elements give the ecosystem its resilience.

Key Concepts for Making Informed Choices | Jeffrey K. Aronson et al, Nature
“Unfortunately, people often fail to think critically about the trustworthiness of claims, including policymakers who weigh up those made by scientists.”
The authors present a set of principles for assessing the trustworthiness of claims about what works, and for making informed choices.

Who Will Lead the ‘Next Frontier’ of Governance Funding? | Catherine Cheney, Devex
“Bilateral donors, under pressure to justify aid spending, are having an increasingly difficult time supporting issues that don’t demonstrate immediate results.”
The author explores challenges associated with funding for governance and points to the beginnings of a promising shift towards coordinated, cross-cutting institutional strengthening to advance development goals.

Promotion Incentives for Public Servants and Improved Service Delivery | Naureen Karachiwalla & Albert Park, VoxDev
“But rather than performance, promotions tend to be based on seniority, education, and connections. This is in part due to the lack of accurate measurements of the quality of service delivery by bureaucrats (Karachiwalla et al. 2019).”
To understand how to incentivize public sector bureaucrats and civil servants to maximize their effort and improve service delivery, the authors studied the promotion system for teachers in Chinese primary and middle schools, and make a number of insightful conclusions about system design.

---

What to Check Out

Guides for Thinking Critically about Claims, Evidence, and Choices
“Side effects of interventions are rarely reported in education. An intervention may, for example, lead to better reading scores but students may enjoy reading less as a result and thus read less.”
A series of short, accessible guides to help you think critically about claims that are made in different disciplines, including agriculture, education, health, the environment, and social welfare.

Evidence-Informed Policymaking Reading List | August 2019

Upcoming Opportunity

Celebrate evidence-informed decision making in Africa during Africa Evidence Week, happening September 9 – 13. This is an opportunity to host a virtual or in-person event, and with the support of the Africa Evidence Network, draw attention to the work being done to advance evidence use in Africa. Suggested themes include: what does the evidence-informed decision-making ecosystem in Africa look like; what difference does evidence make in Africa; and what works for evidence use in Africa?

 What to Read this Month

To Build Stronger Research Institutions in Sub-Saharan Africa, We Need Your Help | Alex Ezeh & Jessie Lu, Center for Global Development

“The ideas that will lead to sustained, transformative change in Africa must and will come from Africans in Africa. Central to this is the building of Africa-led, Africa-based institutions capable of producing research that responds to local needs.”

In response to the current funding system of short-term project grants, the authors offer three models that would better facilitate the growth and capacity building of African research institutions: the multi-stakeholder platform, the integrator organization, and the scale model.

Financing More and Better Data to Achieve the SDGs | The Bern Network on Financing Data for Development

“The commitment to leave no one behind starts with the premise that everyone must be counted. Yet more than 110 low- and middle-income countries under-record or fail to record vital events of specific populations.”

In preparation for the 2020 United Nations World Data Forum on Sustainable Development Data in Bern, Switzerland, this document summarizes multi-stakeholder discussions and proposes a robust framework for funding SDG-relevant data, including a possible new multi-donor Data Financing Facility.

Understanding our Political Nature: How to Put Knowledge and Reason at the Heart of Political Decision-Making | David Mair et al, European Commission Joint Research Center

“The principle that policy should be informed by evidence is under attack. Politicians, scientists and civil society need to defend this cornerstone of liberal democracy.”

Drawing from research by sixty experts in behavioral and social science and the humanities, this report underscores that humans are not purely rational beings, and argues that new insights into human behavior can help address the political crises currently facing democracies around the world. Findings cover how today’s information environment makes citizens vulnerable to disinformation; how better information about citizens’ emotions can improve policymaking; how metaphors and narratives must be used to improve understanding of evidence; and how science can help redesign the way policymakers collaborate.

Does Bottom-Up Accountability Work? Evidence from Uganda | Pia Raffler et al, VoxDev

“The implication seems to be that information-based, bottom-up accountability interventions become less effective as baseline health conditions, utilisation, and the quality of health service provision improves.”

A randomized experiment using citizen report cards and social contracts between communities and health center staff did not improve healthcare outcomes or the demand for healthcare, adding to the body of evidence that citizen engagement, transparency, and accountability interventions may have limited impact on service delivery outcomes.

What to Watch

[Webinar] Mobilising Evidence as a Driver of Social Change for and with Children and Young People | Kerry Albright, for On Think Tanks

“This act of myth busting has been incredibly influential in its own right, so much so that in the countries where we’re working we’re seeing the discourse at government level changing from social protection as a cost to that of an investment, and governments indeed are actually putting in more and more of their own resources to scale up.”

In this webinar, Kerry Albright from the UNICEF Office of Research — Innocenti draws on her experience at UNICEF and DFID to share practical ideas for building an organization’s culture of evidence use, as well as her thoughts on growing youth participation in social change movements.

[Video] Ghana’s Fight to End Poverty with Data | The Global Partnership for Sustainable Development Data 

“Not being counted means you are effectively invisible to government policy. Governments are not going to build a road, or a school, or a hospital for a population that they don’t know is there.”

Watch this inspiring 3-minute video to hear why governments need data to serve their citizens, and see what the government of Ghana is up to.

Evidence-Informed Policymaking Reading List | July 2019

What to Read this Month

Evidence Works: Cases Where Evidence Meaningfully Informed Policy | Bipartisan Policy Center

“The case studies offer unique insights about the challenges that decision-makers face as they grapple with uncertainty, prioritize competing information inputs, and weigh the quality and credibility of available information.”

A collection of 20 case studies at the federal, state, and local level in the United States, tracing how decision makers in government used evidence to shape tax policies, social programs, and more.

Planning Research for Influencing | Oxfam Research Network

“Regularly assess context and make the most of windows of opportunity; use influential people as messengers.”

A handy 1-page planning guide to help you maximize the influence of your research.

New Development: What Works Now? Continuity and Change in the Use of Evidence to Improve Public Policy and Service Delivery | Sandra Nutley et al, Public Money & Management

“More recently, there has been growing support for a more ‘whole systems’ approach to improving evidence use, where systems are viewed as complex assemblages of interlocking networks […] However, a lack of associated practical tools and detailed guidance means that it has been difficult to operationalize these ideas into innovative strategies aimed at improving evidence use.”

The authors highlight 10 key ways in which the use of evidence in policy and practice has evolved or stayed the same over the past two decades. Trends include a growing recognition that simply transferring knowledge from producers to users is not enough; the continued dominance of the ‘what works’ agenda and randomized experiments and the under-valuing of practitioner knowledge; and the potential of experimenting with new ways to structure relationships between evidence producers and users. For a quick summary, click here.

Exploring Research-Policy Partnerships in International Development | James Georgalakis & Pauline Rose, IDS Bulletin

“This Issue aims to identify how partnerships focused on the production of policy-engaged research seek to achieve societal impact and explores the challenges in these processes.”

A compilation of case studies explored through the perspectives of both researchers and their partners in civil society and policy, that explore how research-policy partnerships achieve impact and the challenges they face.

Ghana is Adopting a Data-Driven Approach to Fighting Poverty | Sigal Samuel, Vox

“Recognizing this, Ghana wants its new census data to be more accurate, comprehensive, and granular than in the past. In addition to switching to digital tablets, it’s using satellite imagery to make sure households in rural areas don’t go undiscovered and uncounted, and disaggregating the data it collects at the district level.”

New technologies, increased funding, and a pledge to count the invisible and leave no one behind: preparations for Ghana’s 2020 census signify a strong political commitment to data-driven development, which is growing throughout West Africa.

Data Viz: Global Poverty is Declining – and Few People Know it | Lucas Kitzmüller, IDinsight

“The decline in poverty rates was so steep that also the absolute number of people living in extreme poverty also fell, even while the world population was growing.”

Check out these 3 dynamic data visualizations!

Evidence-Informed Policymaking Reading List | June 2019

What to Read this Month

Reliable, Accurate Information Vital to Policymaking in Africa | Njiraini Muchira, The East African
“The policies are often not implementation ready-they are big in citing evidence to justify why action should be undertaken but weak in using evidence to determine what cost-effective interventions to implement.”
The Executive Director of the African Institute for Development Policy, Eliya Zulu, makes the case for governments to invest in research that informs how to effectively implement policies, and to create training and incentives for government personnel to use evidence in decision making.

Data Roadmaps for Sustainable Development | Global Partnership for Sustainable Development Data
“Interoperability of data and systems continues to be a major barrier for the effective collection, distribution, and use of data, and a barrier for countries trying to set up more modern data ecosystems.”
The seven countries engaged in the data roadmap process have several challenges in common: filling data gaps, sharing data across government agencies and the private sector, increasing political commitment to invest in data, and developing data skills among government officials.

Scaling Social Policy: Five Lessons from Brazil | Megan Dent, Apolitical
“Because Bolsa Família already works with low-income families, the Ministry had instant access to a list of vulnerable households with children under three.”
New antipoverty and social protection programs need a lot of detailed data to identify the poor households that will be eligible for program services; in Brazil, a new early childhood development program was able to utilize already-existing data from a national cash transfer program to target beneficiaries and scale up across the country.

Public Servants and Political Bias: Evidence from the UK Civil Service and the World Bank | Stefan Dercon, VoxDex
“An experiment shows that public servants make errors when interpreting data, incorrectly concluding that it aligns with their ideological preferences.”
The article presents further evidence that even skilled professionals are subject to confirmation bias; peer review / quality assurance units, red teaming, and other strategies could help staff correctly interpret the results, and be more open to results that challenge their preconceptions.

And if you haven’t read it already, check out:
The Politics of Evidence: From Evidence-Based Policy to the Good Governance of Evidence | Justin Parkhurst, London School of Economics
“A heavy focus on individuals as the driving force to improve the use of evidence in policymaking raises two particular issues. The first has to do with the roles of researchers, who are under increasing pressure to ensure that the research evidence they produce is ‘used’ or ‘taken up’. This risks encouraging researchers to have political influence, a role that they are neither trained to do nor one that many feel they have the mandate to take on. The second problem is that such efforts can have a limited duration of impact, given that both researchers and decision makers will naturally change over time or move on from existing positions.”
Parkhurst advances a holistic ‘good governance of evidence’ approach, defined as the use of rigorous, systematic and technically valid pieces of evidence within decision making processes that are representative of, and accountable to, populations served.

---

What to Watch

Ghana’s Infrastructure: The Mystery of Misspending | International Growth Centre

“But just as the problem was hidden within the data, so was the solution”
Still one of our favorite examples of how administrative data can be used to diagnose and overcome challenges to effective implementation of government policies. Well worth a watch!

---

Do you have comments, questions, or ideas for us?
We always love hearing from you! Contact us at info@results4all.org anytime.

 

Evidence-Informed Policymaking Reading List | May 2019

Contribute to a New Blog Series!

How Do We Make Research More Useful? Results for All is partnering with the Global Development Network on a new blog series that aims to capture the point of view and voice of evidence users. See here for more guidance. We welcome all contributions.

What to Read this Month

The Black Market of Knowledge Production | David Mwambari & Arthur Owor, From Poverty to Power

“In order to facilitate swift access to information, the foreign researcher spends money into the research environment, thus creating a peculiar market for knowledge production.”

The authors question the role of money in the production of knowledge in post-conflict contexts (e.g. payments made to interviewees and local researchers), arguing that it can restrict independent local research and exacerbate power asymmetries, and merits further scrutiny.

Can Behavioral Insights Be Used in International Development? | Dr. Kizzy Gandy, Apolitical

“Given the evolutionary roots of human cognition, we have not been surprised to find that people all over the world are prone to the same decision-making biases.”

The author explains that because people share similar human traits and biases, behavioral insights can contribute to tackling complex development challenges in a variety of cultures and economies. She emphasizes the importance of understanding context to tap into the full potential of behavioral insights and describes recent work in low- and middle- income countries.

Evaluators as Conduits and Supports for Foundation Learning | Clare Nolan et al, The Foundation Review

“Foundations and evaluators will better serve the social sector by moving toward a relationship in which evaluators serve as conduits of knowledge that gather and aggregate insights across diverse contexts and organizations.”

Like with governments, philanthropic foundations often struggle to use the findings of evaluations to inform decisions and make programmatic changes. Most evaluations focus on a single foundation or program rather than sharing knowledge across institutions in the same field. This article elaborates on these challenges and proposes solutions alongside examples.

Bridging the Research Policy Gap | Maria Katsonis, The Mandarin

“Colleagues and other federal or state government agencies were cited as the most important sources of research information while internal agency staff were the most frequently consulted source of policy information.”

A survey of 2,000 public servants in Australia provides more evidence that academic research is not a top source of information in government decision making.

Politics Moves Faster than Economists Gather Data | Noah Smith, Bloomberg Opinion

“This episode illustrates how hard it is to base policy on solid evidence when political events are moving quickly and biased, politically motivated actors are grasping at whatever straws they can find.”

In this sobering opinion piece, the author uses the example of recent increases in the local minimum wage to highlight how political decisions must at times be taken before conclusive evidence is available.

What We’re Working On

How can you strategically communicate using evidence to influence policy or program changes, mobilize supporters to take action, or seek input from citizens? We recently facilitated a workshop on communicating evidence for 3ie members using our Agency Roadmap for Building Strategic, Evidence-Based Communications Plans. The roadmap outlines key questions to help agencies identify their communications goals and target audiences, create compelling messages and engage allies, select best-fit tools and channels, and measure the results of communications activities.

What challenges do you face in communicating evidence to key audiences within or outside your agency, and what tools and resources would help you do so? Do you have any feedback on our Roadmap? We hope to hear from you!

Evidence-Informed Policymaking Reading List | April 2019

Invitation to Contribute to a New Blog Series

How Do We Make Research More Useful? Results for All is partnering with the Global Development Network on a new blog series that aims to capture the point of view and voice of evidence users. See here for more guidance. We welcome all contributions.

What to Read this Month

Informing Civil Servants to Make Better Decisions | Daniel Rogger & Ravi Soman, VoxDev

“Informing public policy will require both better research as well as better public sector incentives.”

An interesting study that highlights the important role organizational incentives play in building a culture of evidence use in government. The authors found that when public officials are given authority over decision making, they invest in more accurate beliefs about the constituents they serve, and when they work in an organization that rewards information gathering, they do more of it.

Beware the Well-Intentioned Advice of Unusually Successful Academics | Kathryn Oliver & Paul Cairney, LSE Impact Blog

“Guidance based on best practice and success stories in particular, often reflect unequal access to policymakers, institutional support, and credibility attached to certain personal characteristics.”

The authors explain that the “how to” advice that is commonly offered to academics who seek to influence policy – ensure your research is relevant and of high quality; understand the policymaking process; build relationships with policymakers, etc. – does not address the different contexts and cultures in which policies are made. They argue that this advice helps so long as researchers also have an understanding of their wider role in society and the complexities of the policymaking process.

Unpacking the Issue of Missed Use and Misuse of Data | Robert Kirkpatrick, UN Global Pulse

“We need to reframe how we think about risk in a world of abundant data.”

The author agrees that while robust regulations preventing misuse of data are critical, we also need regulations to ensure that when data can be used for the public good, it always is, and cites several compelling examples to help make this case.

Game: Have You Got What it Takes to Save the Citizens of “Apolitical Land”? | Apolitical

In this interactive policy game, players must work out how to manage spiraling demand on health services in a fictional country.

A quick and fun game in which you, as a policymaker in the Ministry of Health, must choose between options like digitizing services, launching a communications campaign, or adopting an evidence-based policy from another country. Try it out!

Data Science and Programming for Policymakers: A Practical Introduction | James Smith & Padraig Alton, Apolitical

 “The automatic production and refinement of data analyses allow for faster, smarter decision making – and better predictions of, and responsiveness to, events.”

In this short opinion piece, the authors explain why learning about data and programming is important for policymakers, and then offer a short, hands-on course where you can learn and try basic coding.

Sub-prime Evidence: Is Evidence-based Policy Facing a Crisis? | Adrian Brown, Centre for Public Impact

“Perversely then, evidence-based policy is actually preventing us from innovating and collecting any new evidence or insights about what might work.”

Making a parallel to the faulty sub-prime loans that triggered the 2008 financial crisis, the author uses what he sees as faulty and inconclusive evidence behind a U.S. prison visitation program to tell a cautionary tale about evidence-based decisions that discourage further experimentation.

What We’re Working On

In case you missed it, take a look at our recent blog post to read key insights we gleaned from our last year of work, and see what we’re up to next.

Evidence-Informed Policymaking Reading List | March 2019

What to Read this Month

7 Insights for Peer Learning Approaches in Evidence-Informed Policymaking | Abeba Taddese, Results for All

Over the last year, our work at Results for All has focused on exploring how to facilitate opportunities for government policymakers to share knowledge, experiences, and lessons learned in accelerating the use of evidence to inform policy. Here we distill our observations into 7 key insights and principles for peer-to-peer learning for government. Do you agree? What’s missing? Please reply to this email or comment on the blog to share your thoughts!

What Mapping Kenya’s Child Deaths for 50 Years Revealed – and Why it Matters | Peter Macharia & Emelda Okiro, The Conversation

“The findings suggest that, while national trends may be useful for regional and global policy advocacy, they can also be misleading.”

While Kenya has made remarkable progress in reducing child mortality, none of its 47 counties achieved their MDG goals for child mortality. The article highlights research findings that show how national aggregate levels of child mortality in Kenya mask county-level progress. The authors note the importance of setting county-specific targets and collecting data at the subnational level to better achieve and monitor progress in achieving development goals.

The Race to Innovate for Development Should Not Leave Foundational Data Systems Behind | Claire Melamed, Global Partnership for Sustainable Development Data

“Technology and digital advancements provide new sources of data that are invaluable for sustainable development, but we can only take full advantage of these opportunities if core data systems are working well.”

Too many poor people are invisible in the data and numbers that inform government decisions; investing in new data sources and collection systems is essential for policymakers to allocate resources to the people who need them.

7 Types of Policy Makers and What they Mean for Getting your Research Used | Markus Goldstein, The World Bank

“Based on my experiences [with this] there are seven types of policy makers, and knowing your counterpart’s type might be helpful in figuring out how to pitch your discussion.”

The author suggests that by understanding the type of policymaker they are engaging with, researchers can better tailor their approach to meet policymakers where they are, and offers 7 aspects of policymakers to consider in discussions.

Reframing Evidence-Based Policy to Align with the Evidence | Kim Dumont, William T. Grant Foundation

“Studies to date suggest that encouraging evidence-based policymaking approaches that move beyond merely valuing evidence to actually investing in tools and personnel to reconfigure existing routines and practices are likely to yield practices that more consistently map to the evidence and yield better outcomes.”

While government policies and designated funds that incentivize evidence-based programs are helpful, they do not by themselves cultivate the right conditions for evidence to be used systematically to inform government decisions; instead, the author explains that skills, infrastructure, relationships, and trust are essential.

An RFI Guide: How Requests for Information Can Improve Government Human Services Contracting | Results for America

The guide shows how state and local governments are creating stronger, results-focused partnerships that produce meaningful and sustainable outcomes for communities in need. To see some of these recommendations in action, watch this new video featuring the Best Starts for Kids initiative from Washington State, USA and its community-focused contracting strategies to improve equity and outcomes for children and families.

---

Introducing…

Evidence Champion of the Month: Mapula Tshangela

Director of Climate Change Mitigation Sector Plan Implementation

Department of Environmental Affairs of South Africa

 

The South African Constitution requires that the Department of Environmental Affairs (DEA) ensure the right to an environment that is not harmful to the health and well-being of South Africans, in part by mitigating climate change, promoting conservation and biodiversity, and securing ecologically sustainable development and natural resource use. To achieve these diverse and challenging objectives, DEA has built strong partnerships with research entities and strategically invested in generating science and evidence relevant to the environment sector and to both short-term and long-term DEA priorities.

Ms. Tshangela has been at the forefront of this effort, for example, working with colleagues toincorporate new indicators in annual staff performance plans that explicitly link evidence production and use with policy development. In her current role, she is exploring the evidence in climate change mitigation and related action plans by local government and the private sector, together with colleagues from government and academia. “Forming and sustaining trans-disciplinary partnerships such as these has always been key to our efforts to intentional prioritization, gathering, and use of evidence,” she says. Tshangela has a background in chemistry, and symbolically wears a lab coat to work every day to demonstrate the importance of using evidence in climate policy design and implementation.

“Policy implementation change may take a decade or more,

but we can always go back to the evidence we used systematically

and documented to learn from our past decisions.”

---

Do you have comments, questions, or ideas for us?

We always love hearing from you! Contact us at info@results4all.org anytime.

 

7 Insights for Peer Learning Approaches in Evidence-Informed Policymaking

Put yourself in the shoes of a government policymaker for just one day. Your responsibilities for the day are likely to include some management and administrative tasks. Perhaps you will make remarks at a public event or be called on to brief your Minister on short notice. Maybe you will end your day in meetings with external partners and constituents.

Although you may not make important policy-level decisions on a day-to-day basis, when confronted with a policy question, the extent to which you seek out evidence to inform the issue you are trying to address, will depend on your ability to access the information you need when you need it, as well as the expectations placed on you by your agency, constituents, and fellow policymakers. Even with the right intention, you may find it a challenge to systematically use evidence in your work without clear direction or guidelines from your agency on how to find, use, and communicate evidence in policymaking. You may wonder about evidence use in other agencies and direct your staff to look for examples from other contexts that could inspire a new approach in your office, and wish you had opportunities to network and learn from your peers in government.

Results for All’s work over the last year has been focused on exactly this – inspiring and accelerating progress in evidence use by bringing policymakers together to build community and learn from each other. We have highlighted what we learned about peer learning networks through our consultative and reflective research process in earlier blogs and reports, including this mapping of over 50 networks, highlights and a summary report from our peer learning workshop, and this article on the potential of networks in Africa. Here we distill these learnings into 7 insights for peer learning approaches and collaborations that aim to advance evidence use in government.

1. Anchor your evidence use conversation in policymaker priorities. A sweeping conversation about systems, processes, and capabilities for evidence use can lose meaning and feel vague. Instead, identify policy or thematic priorities for your network and use them as an entry point to examine institutional capacity to produce and use evidence. Do policymakers have access to timely, quality, and relevant evidence to inform their policy priority? Are there gaps in technical know-how, analytical ability, or motivation that inhibit a policymaker’s ability to routinely find, use, and report on the evidence that informs this priority? Is there a role for knowledge brokers to play in facilitating increased interaction between evidence producers including the research community, citizens, and practitioners, and policymakers as evidence users, to better inform priorities? What types of incentives could help to facilitate the use of evidence in their specific policies? Start with the problem and help policymakers think about the systems, processes, and capabilities that can be a part of the solution – but keep the conversation rooted in addressing policymaker priorities.

2. Understand policymaker attitudes toward evidence use. Don’t stop with a conversation about challenges or the tangible dimensions of an evidence use culture such as data systems, knowledge management platforms, or evaluations. Go below the surface to deeply understand why policymakers use evidence and if and how it aligns with the way evidence is used in their agency. Explore what it would take to shift underlying assumptions towards greater evidence use – is evidence-informed policymaking perceived as being too complicated, too time-consuming, or too costly? Do staff have the resources, time, and training to find, share, and use evidence? Finally, don’t shy away from engaging on the messy dimensions of the policy process, including politics and power. Take time to understand how these factors facilitate or impede evidence use in order to help policymakers create effective strategies for addressing their policy priorities.

3. Pay attention to practical and experiential learning. Policymakers value opportunities to network and learn from like-minded colleagues. Specifically, we note a high level of interest in tacit “how to” problem solving learning and exchange on the political and process oriented-dimensions of using evidence, to gain experiential insights on questions such as “how did you did you develop a learning agenda in your evaluation plan” or “how did you use evidence to build buy-in for your policy?” We are inspired by the Joint Learning Network for Universal Health Coverage’s approach to drawing on member experiences to co-produce practical tools and knowledge products, and the Collaborative Africa Budget Reform Initiative’s (CABRI) problem-driven approach that facilitates experiential learning in addressing budget reform challenges. Find ways to strike a balance between expert-led presentations and interactive sessions that allow policymakers to draw on each other’s experiences to address evidence use challenges. Finally, ensure there is ample time for networking and building deep connections that can outlast an event or formal platform.

4. Don’t neglect the soft skills. Beyond a technical grasp of the evidence – to find, synthesize, and use complex information in decision making – policymakers also need the skills to effectively communicate research and policy priorities internally across government and externally to their citizens. They also need communication skills to build and sustain relationships with key partners, including the research and academic communities; to engage with stakeholders in building buy-in for a policy or soliciting feedback to improve implementation; and to think strategically about longer-term evidence needs for their policy priorities. While there are many guidelines and toolkits geared towards improving how researchers communicate with policymakers, we have found fewer resources offering practical tips and guidance specifically targeted to policymakers. Engage in conversations to identify gaps in soft skills and find ways to help policymakers improve how they to talk about and act on evidence. 

5. Stay for a while. Relationships and networks play an important role in building a shared identity for jointly problem-solving and co-creating strategies to improve evidence use, and for encouraging the spread of ideas and practices. But it takes more than one engagement to build the kind of trust that leads to an open and honest exchange of experiences. Tacit “how to” knowledge exchange on the softer dimensions of evidence-informed policymaking, in particular, is predicated on trust and close interaction. This type of exchange does not lend itself well to a one-off workshop, so governments, development partners, and funders, please consider long-term approaches for deepening peer-to-peer learning and accelerating progress in evidence use.

6. Implementation challenges keep policymakers up at night.The term policymaking can be misleading in suggesting a focus on making or creating policy, but we have heard repeatedly from policymakers that one of the biggest challenges they face is translating policy to the delivery of services for citizens. Policies that are formed at the national level without considering local-level priorities and an implementation plan are often doomed from the start. Focus on finding ways to support policymakers in taking a systematic and iterative approach to using evidence in both policy design and implementation to avoid this disconnect. A good place to start could be a diagnostic self-assessment process that allows policymakers to deeply explore and understand barriers, opportunities, and strategies for strengthening evidence use practices in policy implementation. 

7. Create a safe space for authority figures and doers.Make an effort to ensure all voices are heard – senior-level policymakers with the authority to approve follow-up activities as well as practitioners and mid-level managers and analysts who will take the work forward. In some contexts, you may need to convene senior-level policymakers and practitioners separately. If your network targets multiple levels of policymakers, pay attention to the dynamic in your meeting room. Have you created a safe and inclusive space that welcomes different viewpoints and perspectives on evidence use?

What’s next for Results for All? With a generous third round of funding from the William and Flora Hewlett Foundation we are exploring how to bring these insights and the resources and tools we are developing to strengthen evidence use in government, to existing global initiatives and platforms that are broadly committed to strengthening policy and practice-level decisions in government. Rather than forming a new network dedicated exclusively to evidence use, this approach enables us to build on existing efforts and to collaborate with partner initiatives and participating governments to improve the use of evidence in their mandates. We welcome your feedback on what we have learned so far and would be interested to know of any initiatives that have an appetite for engaging with us on evidence use. We’ll be sharing regular updates on our new work with partners in our blog and monthly Evidence-Informed Policymaking Reading List, which you can subscribe to by emailing info@results4all.org.

 

 

Evidence-Informed Policymaking Reading List | February 2019

What to Read this Month

Creating a Culture of Evidence Use: Lessons from J-PAL’s Government Partnerships in Latin America | J-PAL

“Allocating even a small amount of resources and personnel to apply the lessons from data and impact evaluations in policy design and implementation, and setting up systems that facilitate this institutional learning, is a crucial part of building a culture of data-driven and evidence-informed decision-making.”

The report draws on interviews with officials from 15 partner agencies and presents key insights for organizations and governments, including: the importance of explicitly making it someone’s job to apply evidence in policy design; creating dedicated spaces where evidence use is rewarded; and investing in administrative data collection and inter-agency data sharing.

Uncovering the Practices of Evidence-Informed Policy-Making | Louise Shaxon, Public Management & Money

“Although the analysis is in its early stages, it does suggest that government departments and agencies concerned to implement a holistic approach to evidence-informed policy-making could consider basing their strategies on seven core practices.”

Drawing on examples from the U.S. (Results for America’s Invest in What Works Federal Index) and the UK’s Department for Environment, Food & Rural Affairs (Defra), the paper identifies a minimum set of practices to help governments take a holistic approach to evidence-informed decision making.

Shifting Our Mindset on Financing Data for the Sustainable Development Goals | Sarah Lucas, the Hewlett Foundation

“Most human beings (except us few data evangelists) are not personally moved by a long list of data gaps. But they can be deeply moved by stories of how data help solve problems they care about.”

On the eve of a global donors meeting on financing for data for the SDGs, the author encourages participants to think like a data user and take a portfolio approach to data investments.

Data Impacts Case Studies | Open Data Watch

Looking for stories of how data help solve problems? Consider these 5 short, to-the-point examples:

• Public Data Lowers Violence in Cali, Colombia
• Mapping Commuters Streamlines Transit in Nairobi, Kenya
• Using Satellite and Cell Phone Data to Eliminate Malaria in Namibia
• Climate Modeling Helps Colombian Farmers Improve Decision Making
• Better Data Improves Women’s Lives

 Building a Culture of Research Impact | Louise Shaxson, LSE Impact Blog

“But we think that what [the four cases] all had in common was a culture that emphasised, above all, responding to local needs.”

In this blog, the author shares insights from case studies that were commissioned to understand how researchers engaged with decision makers and the impact of these efforts. The author notes that a common emphasis on addressing local needs in all the case studies contributed to successful research partnerships, and shares additional insights for achieving research impact, including the importance of taking an iterative approach, networking, and ensuring the quality of evidence.

---

What We’re Working On

Read how global development funders are supporting governments to use data and evidence in policy decisions in our latest report. This rapid review summarizes insights from our interviews with 23 bilateral, multilateral, and philanthropic funding organizations to understand how global development funders are investing in evidence-informed decision making in government. It discusses the constraints governments face in promoting the systematic use of evidence, what funders are doing to help address these constraints, and what else is needed to build a broad culture of evidence use in governments of the Global South. Its aim is to inform a conversation among development partners that catalyzes collective action to strengthen evidence use in government. Join the conversation by sending us an email at info@results4all.org. We welcome your ideas and comments!