UNICEF’s work in Serbia aims to develop and embed effective programs that address disadvantage and child rights locally and to influence systemic improvement. Drawing on a team across UK, Singapore, and Australia, CEI and our Serbian-based partner SeConS Development Initiative Group carried out an evaluation of UNICEF Serbia’s strategies for program scale-up and an assessment of the scalability of five early childhood programs at different stages of scale-up.
Our team found that though the programs are well regarded, there are significant friction points with current services and community cultures. Recommendations included that UNICEF:
(1) design programs to be better aligned with current systems and
(2) plan a wider program of work to narrow the gap between intended and actual system performance. CEI presented findings to regional UNICEF leaders and recommendations are now being adopted.
The project began with an inception mission in Belgrade in early 2020. The team synthesised key literature on effective scaling-up, with a particular focus on low- and middle-income countries, and then built a new analytical framework, styled on the UNICEF Scale-up Framework. This evidence-informed framework set out key criteria for scalability in four domains:
(a) an optimised program, which is
(b) fit for context,
(c) supported by evidence, and has
(d) secured commitment for scale-up from key stakeholders.
The framework was then used to assess the scalability of the five programs, and to advise on further work needed.
The evaluation drew on extensive analysis of program and planning documentation, engagement with UNICEF teams, qualitative interviews with over 50 national and local stakeholders (representatives from government ministries, national sector bodies, partner NGOs, research groups and local implementation partners) and a survey of local implementation sites. It also drew on the Consolidated Framework for Implementation Research [i] and on the Dynamic Sustainability Framework.[ii] The latter was particularly valuable in highlighting the need for iterative activity to narrow the gap between current and desired system performance.
A key message that emerged in the evaluation was the importance of programs fitting with policy contexts and priorities. Programs that straddle multiple policy portfolios and/or challenge the social status quo face a relatively greater challenge to be embedded in their systems – and therefore, to be scaled up. Tension also exists between the delivery of important programs (such as the five in the evaluation), alongside the need for improvement of the system in which they are operating to better support delivery and scale-up. Designing a program to fit the current system may not achieve wider ambitions – but if systemic change is needed to give the program the best chance to flourish then this is an immediate challenge to scale-up. This dilemma is not unique to Serbia; it is faced by service systems, practitioners, and policymakers globally.
CEI made eight recommendations to strengthen UNICEF’s program development and testing and approach to scale-up and set out recommended next actions for taking the five programs forward; these recommendations are all now being implemented.
The final report is now published.
Notes:
[i] Damschroder, L.J., Aron, D.C., Keith, R.E. et al. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implementation Sci 4, 50 (2009). https://doi.org/10.1186/1748-5908-4-50
[ii] Chambers, D.A.,, Glasgow, R.E. and Stange, K.C. (2013) ‘The dynamic sustainability framework: addressing the paradox of sustainment amid ongoing change’ Implementation Science 8:117