Lessons learnt from Realising Ambition
Replicating evidence-based interventions for children and young people at risk of entering the criminal justice system isn’t easy. However, it’s exactly what the Big Lottery Fund’s Realising Ambition programme is doing.
The latest evaluation report (PDF) unpacks some of the biggest challenges, from the perspective of the consortium delivering the programme.
The Young Foundation’s Tricia Hackett is closely involved in supporting Realising Ambition projects through the replication process. Here she takes us through the top five early lessons we’ve learned.
1. There aren’t enough evidence-based programmes to choose from.
During the application phase it was explicit that only applications from well-evidenced interventions would be considered for the programme. Over 240 interventions applied yet only four percent met the Social Research Unit’s highest standards of evidence. The pool of interventions underpinned by the highest quality of evidence is too narrow and leaves limited choices for funders and commissioners.
2. Organisations tend to overestimate the demand for their service.
Early indications show that organisations might have had an unrealistic estimation of demand and this has meant that there is a shortfall in meeting planned delivery numbers. Sometimes projects have struggled to engage the eligible children and families because there is insufficient or poor data about the level of need or the distribution of need in the target areas.
3. It is difficult to know which promising interventions will be able to deliver impact.
Whilst ten of the interventions are tightly defined and underpinned by the highest standards of evidence, the remaining interventions have a range of evidence including questionnaires, on-going quality assurance and qualitative case studies to offer preliminary evidence of impact. These methods won’t necessarily give commissioners and funders the degree of confidence that they need.
4. It is easy to conflate evidence-based programmes with flexible practice…and it is harder to faithfully replicate practice.
Some interventions are processes which can be broken down into discrete elements and sometimes can be hard to differentiate from programmes – interventions as processes are common in the public service systems.
Processes are more difficult to evaluate and lend themselves to more flexible practice which can be at odds with the level of fidelity that is needed for successful replication.
5. Things tend to take longer and cost more than originally planned.
The time and resources required to deliver the interventions (e.g. recruitment, staff training, engaging a new set of stakeholders, navigating internal structures in larger organisations and setting up new structures in smaller organisations) are often underestimated.
These are some of the early learning points from the first year of Realising Ambition and we expect that many more will unfold under the course of the five-year programme so watch this space.
Tricia Hackett is a Programme Leader working in Applied Innovation at the Young Foundation.
What do you think of Tricia’s guest blog? Leave your comments below or join the converation on Twitter using #biglf.