In this article, Jonathan Breckon, Director of The Alliance for Useful Evidence in the UK, joins CEI Executive Director Dr Robyn Mildon, to explore common issues for those considering setting up a new evidence centre and ideas on how to surmount them.
This article first appeared in The Mandarin on April 26, 2019.
A guide to setting up a What Works evidence centre and surmounting its challenges
Since 2010, the UK has built a network of What Works centres, with the purpose of championing ‘what works’ across the UK and ensuring that spending and practice in public services is informed by the best available evidence.
These 10 independent centres — dedicated to evidence synthesis, generation, transmission and adoption — function as knowledge brokers and key advisors to government and other organisations for their areas of policy focus.
The UK What Works Network represents one of the first attempts to take a national approach to prioritising the use of evidence in public policy decision-making.
The evidence-informed decision-making agenda is not new to Australia, reaching as far back as a decade ago with the Productivity Commission’s 2009 Annual Roundtable on “Strengthening Evidence-based policy in the Australian Federation”.
Recent announcements by the federal Labor Party – such as the promise to establish an Evidence Institute for Schools and an ‘Evaluator-General’ in the Australian Treasury – are designed to improve the use of evidence in Australian policy and spending decisions.
The 10 centres comprising the UK What Works Network.
In considering the What Works Network concept in the Australia context, we can learn a lot from the UK experiences, successes and failures.1
In setting up any new evidence centre in Australia, the challenges below will need to be addressed. By addressing them from the outset, the centre will have the best chance of success in creating real and sustained change in the way evidence is used in public policy decision-making.
Mobilise your knowledge
The biggest single challenge for any What Works-type organisation is having the evidence base ignored – ensuring that its evidence base is actually used in practice is fundamental to the success of a what-works centre.
Part of the aim of a What Works centre should be to speed up the “research to practice” gap, which requires active knowledge mobilisation and implementation support to be incorporated as part of the model.
A recent report in the UK identified a lack of focus by the UK What Works centres on supporting the adoption of evidence, with many of the centres having recognised this and now moving to increase their production of actionable guidance and supporting the uptake and application of evidence.
As well as embedding this type of support, any What Works centre needs to actively engage its target audiences. Many institutions that the Alliance for Useful Evidence has spoken to regretted not putting enough resources into communicating and marketing at the start of their work.
For instance, in its early years, the U.S. Department of Education’s What Works Clearing House focused too much on the excellence of its research, and not enough on communication. It has now moved towards a stronger emphasis on ‘usability’ and ‘educator-friendly products’.2
Avoid weak evidence
Sometimes there is not enough evidence out there for a What Works centre to review. One UK centre was nicknamed the ‘What Doesn’t Work Centre’ because it only found negative results, or poorly designed evaluations.
If this is the case, it’s important to fill the gaps and do some original, well-designed, primary research that helps to build the evidence base.
The UK’s What Works centres have approached this in a number of ways.
For example, The What Works Centre for Children’s Social Care is running randomised control trials, The Education Endowment Foundation tests the impact of high-potential projects to generate new evidence of ‘what works’, and The Centre for Homelessness Impact has published ‘Evidence Gap Maps’ to show the research community what needs to be done.
Find sustainable funding
The third core challenge in any What Works-type organisation is funding. The amount, duration and sustainability of funding heavily impacts the agenda of the What Works centre. Without a sustainable and long-term funding source, centres can waste lots of time chasing grants from government or not-for-profits.
Funding structures for the UK What Works centres vary quite significantly. The ideal is to have a core of funding over a long period, such as the Education Endowment Foundation’s £125m founding grant from the UK Department for Education, which has allowed the Education Endowment Foundation to take a long-term strategic approach to its work and contribute significantly to the generation of a new evidence base.
Compare this to the 17 Whitehall agencies that fund the What Works Centre for Wellbeing.
Avoid academic capture and foster independence
A What Works centre needs to steer clear of being just another university research body.
The centre is there for other audiences, such as teachers, police officers, social workers and policymakers, not for academic advancement alone.
That doesn’t mean that the centre should avoid locating inside a university – some successful ones have found homes inside higher education, including in the LSE (What Works Centre for Local Economic Growth) and in Cardiff University (the Wales Centre for Public Policy).
But if a centre is based in a university, it must foster an independent-minded culture, with an eye beyond the churn of academic publishing.
Similarly, independence from, yet deep links to, government is one of the key factors that has led to the success of the UK What Works centres. It has allowed the centres to build long-term research agendas set aside from government priorities and Ministerial influence, while also remaining close enough to be able to integrate their work into policy.
Hire brilliant staff
Finally, one crucial element is making sure a centre has good leaders. Leaders need to be credible with a range of audiences.
To give an example, the energy and entrepreneurialism of Rachel Tuffin at the What Works Centre for Crime Reduction has been vital in advancing that institution. Another example is David Halpern, the What Works National Adviser, based in the UK’s Cabinet Office. He is just as comfortable getting the ear of ministers, as he is speaking with professors.
Nobody believes that ‘evidence speaks for itself.’ It needs leaders who get out and make sure evidence is used at the right time and place in the right way.
As the What Works centres mature and grow in the UK, and the movement spreads internationally, there will be more lessons to be learned and shared.
We also hope there will be greater opportunities for international ‘what works’ collaborations to enhance the use of evidence in key areas of social policy to improve outcomes for beneficiaries everywhere.
References
- These recommendations are based in part on interviews with leaders in What Works-type centres, a review of the ‘grey literature’, and evaluation of evidence intermediaries, as part of a study carried out by The Alliance For Useful Evidence for the Nuffield Foundation, a UK non-government organisation. The ideas also reflect the authors’ own anecdotal experience, particularly Jonathan Breckon’s experience in incubating the What Works Centre for Children's Social Care.
- United States Government Accountability Office. (2013). “Further Improvements Needed to Ensure Relevance and Assess Discrimination Efforts.” Report to the Committee on Education and the Workforce, House of Representatives, p.12. Available at: https://www.gao.gov/assets/660/659425.pdf.
Update
More recently we have have undertaken a review into major, system-level efforts to cultivate evidence-based practice: an assessment of ‘what works for what works,’ with the objective of understanding the successes and failures of evidence institutions and intermediaries, focusing in particular on the education sector.