Getting to grips with replication
At the end of last month, BIG convened a group of forward looking trusts and foundations for the first of three seminars on ‘Getting to Grips with Replication’. Alongside Geoff Mulgan of NESTA and Stuart Thomason from the Young Foundation, Tim Hobbs spoke to the group about his experiences of replication working at the Social Research Unit, Dartington. His guest blog reflects upon just a few of the key themes emerging from the rich discussion…
Not everything needs a robust evidence-base before you replicate it
There are some social interventions or services that you don’t need a robust evidence-base for prior to replication. These include, for example, safe and inviting playgrounds for young children in urban environments or schemes to get food, clothing or warmth to those desperately in need.
But there are many social interventions where robust evidence of impact is arguably a pre-requisite of replication and scale. The last thing anyone wants to do is propagate ineffective or harmful interventions.
Let’s face it, whilst almost all social innovations and services are well-intentioned, the efficacy of most is unknown despite the protestations that “we just know it works”.
But too few services or interventions are well-evidenced and ready for replication
Even in the relatively well-developed world of children’s services, there is not too much for a funder or commissioner to choose from and replicate that is well-evidenced. Of the 1,100 interventions reviewed by Blueprints for Healthy Youth Development – the leading database of evidence-based programmes in the US – only 44 meet the Social Research Unit’s high Standards of Evidence.
Of the 240 applicants to the Big Lottery Fund’s Realising Ambition programme, just 10 made the highest grade (for the remainder of the funded projects the evidence was slightly below this level). In both cases, this equates to just 4%. What’s more, this 4% figure relates to streams of work explicitly reaching out for well-evidenced interventions. The ‘real’ proportion of services with a robust evidence-base is certainly less than 1%.
What could funders to do improve the Innovation to Impact Pipeline?
There is a lot that a more coordinated effort from trusts and foundations could achieve in terms of strengthening what we at the Social Research Unit refer to as the Innovation to Proven Impact Pipeline. Here are some ideas discussed at the seminar:
- Map the sectors they invest in, plot the degree to which their individual and collective investments are innovative or evidence-based and the scale to which such services are implemented
- Coordinate – across Trusts and Foundations – where investments are made across the Innovation to Impact Pipeline, contributing to a more balanced spread of investment across the pipeline
- Invest in the tools and support structures that organisations require in order to refine interventions and ensure they are implemented and replicated with fidelity
- Invest less in light touch evaluation of social interventions that tell you little about impact, and more in robust experimental evaluation to help build the evidence base across the UK, plus smart data dashboards allowing real-time monitoring and evaluation allowing more reflective adaptation and testing of interventions.
The time has never been better for greater coordination between Trusts and Foundations. I hope this seminar series – and the conversations and actions that result from them – lead to a more functional innovation to proven impact pipeline.
Tim Hobbs is Post-doctoral Researcher, Social Research Centre at Dartington
What do you think of Tim’s guest blog? Join the discussion on Twitter using #BIGreplication or leave your comment below.
The second of three ‘Getting to Grips with Replication’ seminars will be held on Wednesday 13 March, and and the third on Thursday 21 March. There are a handful of spaces left – email firstname.lastname@example.org if you would like to attend.