Centre for Evidence and Implementation, along with co-hosts Monash University, held our bi-annual Evidence and Implementation Summit on March 30 and 31. The Summit is one of the leading sector events on the global calendar, among only a handful of gatherings that focus on evidence synthesis, evidence translation, evaluation and implementation science. More than 750 participants from 30 countries joined in, and both the online EIS Portal and the conversation on Twitter at #EIS2021 enabled this global network to connect in real time.
This year’s focus – ‘Bringing together evidence, implementation, policy, and politics: Towards a better future for all’ – came to life through 10 keynotes featuring leading voices, alongside numerous panels, discussion groups and workshops.
In this month’s newsletter, we explore memorable Summit moments through the lens of its five sub-themes. A selection of content-rich Twitter threads is linked below and registrants can access all of the cited sessions through the online EIS Portal. (For details on how to access EIS content, see the closing paragraphs of this newsletter).
Fast, good, cheap - choose two! Generating and implementing rigorous policy and practice-ready evidence
Decision-making and public debate are best served if policymakers have access to the best current evidence on a given issue. Understanding how to meet this need was a key theme. In his keynote, Professor John Thwaites of Monash University shared challenges in implementing evidence in political settings: “often we can have too much, rather than too little evidence; the need for discernment is great.” As such, specialist institutions can be powerful allies in evidence translation.
Enabling evidence use within government is another approach: in her keynote, Director of the Africa Centre for Evidence Dr. Ruth Stewart described how the South African Presidency has embedded an evidence-informed approach to policymaking through the Socio Economic Impact Assessment System, which facilitates assessment of the socio economic impact of all draft policies, bills and regulations.
An invited panel comprising Professor Annette Boaz of the London School of Hygiene and Tropical Medicine, Chris Webber of the UK's Open Innovation Team and Professor Becky Francis of the Education Endowment Foundation presented ideas to increase the implementation of evidence in policymaking, including the use of rapid response working papers, increasing funding for skilled intermediary individuals and organisations, co-production and a focus on systems to drive true impact.
Context counts: translation, adaptation and transportability of evidence across people, place, culture and systems
Several sessions addressed approaches to adapting evidence for context. Panel 8, “Contextual relevancy: Educators' early insights into quality use of research in practice”, featured a number of speakers including Associate Professor Mark Rickinson and Dr Jo Gleeson from Monash University who showcased the Q Project, a five-year Australian study that seeks to understand and improve the use of research evidence in schools. The study has found that teachers and school leaders report contextual relevancy as a major influence on their sourcing, selection and use of research.
Context was also the topic of Discussion Event 8, moderated by CEI’s Managing Director Mary Abdo, where contributors shared studies from the UK, Ireland, South Africa, Kenya, Switzerland and Finland. J-PAL detailed its involvement in evaluation of Teaching at the Right Level (TaRL) a well-evidenced learning approach developed in India, then brought to sub-Saharan Africa. In England adaptions to the Family Nurse Partnership programme enabled the implementation of service delivery improvements at scale across the country.
Core competencies for evidence synthesis, translation and implementation practice and science
Many presenters discussed the research knowledge and competencies required to access, synthesise, analyse, and interpret evidence as well as the skills required to implement this evidence in practice. These skill sets encompass research literacy alongside relationship-based competencies like effective communication (e.g., with policymakers), leadership (including the capacity to develop and empower teams, demonstrate vulnerability, and address power differentials) and the ability to cultivate strong internal and stakeholder relationships based on trust.
A few sessions in particular took up these themes. CEI Associate Dr Bianca Albers introduced Professor Sue Dopson of Saïd Business School at Oxford who spoke on leadership for transformational change. Addressing implementation in complex settings like healthcare, she encouraged leaders to hold uncertainty for long enough to really understand problems, in order that solutions properly fit.
Dr Allison Metz of the National Implementation Research Network ran a pre-Summit workshop with CEI Executive Director Dr Robyn Mildon on “Competencies needed to support implementation and evidence use.” The workshop focused on three domains including stakeholder engagement and co-creation, ongoing improvement and sustaining change. Dr Ruth Levine of IDinsight expanded on the theme in her keynote, stating the best way to engender trust is through co-creation: the intentional and ongoing engagement of those affected by research.
Innovation in implementation science, evaluation design, evidence synthesis, scaling up
Presenters shared a range of methods and practice innovations designed to make sense of context, to reflect the complexity and dynamism of diverse settings and circumstances, and to ensure that both the implementation and the effectiveness of programs and practices are assessed in evaluations.
The first Invited Panel of the Summit, “Implementation Science and Behavioural Insights: Better together than apart?” featured Associate Professor Rinad Beidas from University of Pennsylvania, Dr Alex Gyani from the Behavioural Insights Team, CEI’s Executive Director Dr Robyn Mildon and Dr Joanne Yoong from University of Southern California/Research for Impact. The group agreed that implementation science is a team sport and that behavioural science has enabled implementation scientists to share a language of behaviour change with policymakers.
In Panel 22 CEI Director Dr Vanessa Rose, CEI Senior Advisor David Taylor and Professor Aron Shlonsky of Monash University described how a hybrid implementation-effectiveness design can be used to evaluate programs, which followed on from a pre-Summit workshop offered by Professor Geoff Curran of University of Arkansas and Professor Shlonsky that trained participants in the use of hybrid design methodologies.
In his keynote David Cowan, President of the Australian and New Zealand Society of Evidence Based Policing, described how behavioural science and evidence-informed practice are improving criminal justice outcomes by enabling police to identify ‘ingredients’ of crime and to be proactive about preventing a resurgence of crime post-COVID.
Keynote speaker Professor Bruce Chorpita of the University of California described innovation in the use of common practice elements and shared his vision of making all mental health evidence actionable in any context, challenging the audience to "imagine an evidence-base that can implement itself.” The subsequent Invited Panel on “Common elements in practice” examined innovations in approaches. Presenters included: Professor Bryce D. McLeod of Virginia Commonwealth University, Dr Aleisha Clarke from the Early Intervention Foundation, Dr Patty Leijten of Research Institute of Child Development and Education, University of Amsterdam, and Jane Lewis, CEI’s Director for UK & Europe, moderated by Associate Professor Rinad Beidas from University of Pennsylvania.
Building a better future: Systems change for complex problems such as poverty, inequality, climate change, mass migration
A number of presenters urged the sector to seek strategies that can deliver change at the systems level. Dr Jenny Donovan, CEO of the Australian Education Research Organisation, noted in her keynote that we need to shift our focus to systems, zooming out from the study of individuals to instead consider the structures and policies that drive marginalisation. CEI Senior Advisor Dr Ellie Ott introduced Professor Mary Dixon-Woods, who noted in her keynote on improving health care systems that we must broadly interrogate how problems are identified, defined, and selected for attention – by whom, through which power structures, and with what consequences.
In her keynote Associate Professor Rinad Beidas described a “system driven, iterative approach” in Philadelphia, backed by its Commissioners of Mental Health, to making evidence-based practices available to everyone in the system as a matter of social justice. This ensures Beidas and her collaborators access to a “natural lab” for a significant umbrella study, including the barriers and facilitators to practice change – for example finding that strong organisations (e.g., those with proficient organisational culture) were correlated to use of evidence-based practices.
CEI Director Dr Jessica Hateley-Browne participated in Panel 17, which looked at an evaluation of the NSW Government’s Future Directions for Social Housing Policy. A consortium used a hybrid study design, with a dual focus on assessing effectiveness and implementation that ultimately aims to improve the quality of evidence available to inform social housing policy and practice.
Sandy Pitcher, Secretary of the Department of Families, Fairness and Housing in Victoria, used her presentation to detail how increased access to linked data sets combined with implementation science is providing new insights on how to target programs to those most in need, enabling reforms across human services and improving system-wide outcomes in child social welfare. She highlighted the importance of evidence use at all stages of the policy cycle and the delivery of consistent services that also match individual need.
Thanks are due to our Gold Sponsors, Behavioural and Implementation Science Interventions at National University of Singapore, and the Department of Families, Fairness and Housing in the Victorian State Government along with the partners listed below, without whom this exceptional event would not have been possible.
3iE: International Initiative for Impact Evaluation
Analysis and Policy Observatory
Australian and New Zealand Society of Evidence Based Policing
Department of Communities and Justice, Government of NSW
Institute for Safety, Compensation and Recovery Research
National Centre for Implementation Science
Research for Impact
Save the Children
We encourage you to continue to engage with the rich Summit content at the Portal, where it will be available until March 2022. If you didn’t register for EIS and therefore don’t have a Portal login, please email Event Manager Nikki Abercrombie for access rates and conditions.
The next Evidence and Implementation Summit will be held in October 2023! Sign up to be notified as details become available.