Building Skills and Confidence – Supporting Change and Impact Evaluation
Rocket Science was appointed by the Big Lottery Fund to evaluate a £60 million fund designed to build the skills and confidence of around 1500 grant-holders. Supporting Change and Impact funding was developed to support rant-holders by helping to improve their sustainability and their impact. The fund was split into two elements, Supporting Change – a small grant up to £10k focused on organisational development support and Supporting Impact – continuation funding for up to one year. We were asked to design a method that captured the experience of projects as they sourced and benefited from support to inform BIG’s policy on Building Capabilities and Impact.
Drawing on our experience of delivering three large evaluations for BIG, including the Awards for All programme and Community Asset Transfer (England and Wales) we designed research tools to help projects develop their reporting skills which encouraged and secured participation, whilst keeping the costs of primary research down. The methodology was designed to capture information on a number of levels, so that we could provide answers to key research questions; what should BIG consider in developing its approach to building the capabilities of organisations in order to deliver better against their outcomes and support their sustainability? How can BIG identify and support excellence in future? This comprised a three phased approach: Phase 1 included a baseline survey of all funded projects, a review of policy and practice around impact and sustainability, interviews with internal and external stakeholders and a series of case studies of funded projects to understand in more detail how projects were using their funding. We used an on-line survey which captured high-level data which we able to revisit in order to analyse trends in Phase 3. Phase 2 focused on gathering further insights into how other funding arrangements worked and explored this practice with detailed case studies with five lead partners. We also developed the Supporting Change and Impact Drilldown tool (SCID) to capture more detail from a mixture of projects and produce a baseline report. Phase 3 included a revisit of the baseline survey, telephone interviews with a selection of projects to explore how the funding has supported the organisation twelve months on and follow up research using the SCID tool to track the impact of the funding.
Our findings concluded that Supporting Change and Impact was a timely, well-regarded resource for existing grant holders. Although some projects may have closed, organisations were still on their journey towards greater sustainability, others were using the outputs to promote their services to funders, stakeholders and their communities. The diversity and nature of services delivered through BIG grant-holders reflected the different ways in which the funding was used. Flexibility in determining how to diagnose and source support was really important, allowing projects to tailor support to their own circumstances. But, it had implications for BIG in how it took forwards its policy on building capabilities, in particular the extent to which it wants to promote support to grant-holders. Evidence suggests that organisations already have access to the support they need, and do not readily use directories or other tools to source this.
Early findings also suggested that Supporting Change could become a universal grant, built into the award at the outset and avoiding the need for an application and associated administration costs. Throughout the evaluation we involved funding and policy staff in the design, planning and review of the evaluation. This was effective in enabling live learning and implementation, as well as ensuring that findings from research were translated into action.
Contact Caroline for more information
VCS Assist – Building the skills of VCS to secure ESF and Big Lottery Funding
Voluntary, community and social enterprise organisations face increasing challenges to maintain high–quality employment support for customers with complex needs, in a commissioning environment which is driving unit costs down and incentivising lead/prime contractors to focus on the most employable at the expense of the harder and more expensive to help. Drawing on our primary research and evidence of previous programme evaluations, our recommendation to refocus future capacity building support was accepted by London Councils and the European Programmes Management Unit (EPMU), responsible for ESF in the capital. In 2012, Rocket Science was part of a consortium of organisations, including London Voluntary Service Council (LVSC), London Training and Employment Network and the Black Training and Enterprise Group which successfully put forward a new model of technical assistance for third sector employment-related service providers.
The “VCS Assist” partners identified a range of skills and cultural changes required to ensure that providers, whatever their size or specialism, have the opportunity to adapt to the new climate and continue to meet demand for their services. The new skills needed include: articulating an offer (which may include creating delivery partnerships/consortia), pricing services competitively, developing and understanding client pathways and pitching directly to lead contractors, and delivery partners; contract negotiation and managing cash-flow in a payment–by-results system.
We are continuing to provide support and training to hep organisations prepare for the new ESF commissioning rounds 2014-2020 and working with the London Voluntary Service Council in the build up to the Building Better Opportunities Fund from the Big Lottery Fund.
For more information contact Caroline or John
Cost benefit analysis of NEET early intervention programmes – West London Alliance
Rocket Science has been working on the Youth Strand of the West London Alliance’s 6–borough “Whole Place Community Budget” pilot. We conducted a ‘deep dive’ review into the provision and support for 16-18 year olds in West London, the barriers and challenges, what works and the implications for programmes to reflect requirements for Raising Participation Age. Our research led us to a new model of early intervention developed by Impetus PEF (“Think Forward”) designed on an “invest to save” basis to prevent young people becoming NEET by offering those identified as most at risk a dedicated life coach. A 2-year pilot of 5 coaches in the London Borough of Tower Hamlets had enjoyed a 98% success rate – only 6 of 320 teenagers referred to the programme did not progress into post-16 learning or work. Tower Hamlets was nevertheless unable to continue the model owing to budget constraints.
The Private Equity Foundation (PEF) was at the time looking to scale up a programme to reduce NEETs, and to fund Think Forward through a Social Impact Bond. Rocket Science worked with Impetus PEF and the West London partners to identify ways to replicate the Think Forward model in the context of Whole Place Community Budgeting. In making the business case for a new preventative approach to tackling NEETs, Rocket Science consultants were trained in how to use the New Economy Manchester Cost Benefit Analysis toolkit which enabled us to quantify the anticipated cashable savings over 5-10 year of more effective early intervention with young people showing a clutch of RONIs (Risk Of NEET Indicators) in years 6 and 7.
We assessed the potential impact of a proposed West London pre-NEET 14-16 intervention. This involved 100 young people selected as being at risk of NEET supported by a dedicated coach for the duration of the programme (up to 5 years). The aim of the £450,000 intervention is to reduce the numbers of NEETs in the target area/group of schools by 50%. We estimated, from previous pilots, that the outcomes would be:
- Reduction in numbers moving onto JSA at 18 – 55%
- Reduction in people claiming ESA/IB by 50%
- Reduction in numbers claiming LPIS by 60%
- Increase in level 2 qualification by 60%
- Increase in level 3 qualification by 40%
- Reduction in mental health interventions by 20%
- Reduction in ASB committed by the cohort by 25%
- Reduction in all crimes committed by the cohort by 25%
- Reduction in safeguarding by 20%
- Reduction in drug and alcohol abuse by 25%
- Improvements in individual family and community well-being by 40%
- Using the New Economy Manchester Cost Benefit Analysis toolkit we assigned monetary values to outcomes finding that the cost benefit overall was:
- A fiscal benefit cost Ratio of 1.64 (£745,000 in benefits)
- A economic benefit cost ratio of 0.83 (£380,000 in benefits)
- A social benefit ratio of 1.86 (£848,000 in benefits)
The fiscal and social benefits indicators are positive over a five year payback period. The economic benefits are lower owing to the age of the young people involved and the fact that they are not in the work place for several years of the proposed CBA timeframe.
Contact John for more information
Reviewing early intervention and service effectiveness – City of London
Rocket Science was selected to research alternatives to the City of London’s commissioning of early intervention services for residents. The pilot focused on the Portsoken ward (dominated by 2 housing estates) and was intended to inform the Corporation’s Health and Wellbeing Strategy in preparation for their new role in public health. It was also intended to identify ways in which services could be better coordinated and avoid duplication, through person-centred commissioning.
An initial phase involved extensive mapping of and consultation with key stakeholders, from both within and outside the City, to inform our approach for gathering spend and use data. We conducted an extensive desk review of existing public data on the ward, which was updated periodically to reflect new Census data as this became available. NOMIS data was used to analyse deprivation, and how the ward compared to the City, London and UK. Analysis of tenancies showed that the population has remained quite static, in part a consequence of the high level of services provided by the City, but also a reflection of strong cultural ties as residents chose not to move despite highly overcrowded housing conditions. Indicators of health and economic wellbeing were at best static or in decline and, in spite of recent investments in public services in the area, the anticipated improvements to the area were not being realised.
Our research was corroborated by the experiences of residents and users of a range of services, drawn from a series of interviews, mini-focus groups and a survey of providers. Whilst some services were highly valued, they were poorly coordinated and suffered because of low public awareness and insufficient promotion. We presented the findings to both members and stakeholders and ran a consultation process from January through March 2013 in order to co-design recommendations in the light of a number of key policy changes. These included on-going welfare reform – residents were perceived as at risk through digital exclusion, language barriers and the inaccessibility of better paid employment, and consequently the ward continued to have high levels of ‘working poor’; the Public Services (Social Value) Act 2012 – what added value could be provided through better procurement, and the Localism Act 2011 – how could community empowerment reduce local dependency on public services?
The recommendations from the review which focused on improved asset management and developing community capacity have been adopted in full. Key to the future of the ward is the development of a residents’ and providers’ forum which will become central to the design and commissioning of services. A capacity building programme has been put in place over the next 12 months and an innovative community currency, Time Credits, to transform community engagement from passivity to pro-activity in the design and receipt of services.
“The Portsoken All Ages Early Intervention Review has a compelling vision for the future delivery of services in the area and some very positive recommendations about using our assets more effectively and delivering better health, welfare and employability services.” Neal Hounsell – Assistant Director Commissioning and Partnership.
For more information contact Caroline
VCS Assist support for Camden Society
Through the VCS Assist programme we helped the Camden Society understand and demonstrate their impact they were having in securing employment for people with disabilities across all of their programmes. This was to help them demonstrate that their performance was far higher than national programmes in supporting disabled people into work as well as capture the experience of employers and trainees. We produced an infographic, both static and web enabled using our Piktochart software to help them spread their message and impact to funders and prime contractors. You can view the online version here.
Contact Caroline to find out more about our impact and visualisation work
Federation of Small Business
Rocket Science were commissioned by the Federation of Small Businesses to explore the employment potential of micro-businesses (employing fewer than 10 people) and the extent to which there was scope to enhance recruitment by micro-businesses by providing appropriate recruitment-focused support. We reviewed statistical evidence from across Europe and our findings included:
- Micro-businesses make up nearly 94% of Scottish businesses and provide 27% of the private sector jobs in Scotland.
- This underplays their significance in tackling unemployment:
- Over 40% of unemployed people who find work in the private sector go to work in a micro-business or become self-employed.
- 26% of recruits to micro-businesses across the EU are long-term unemployed people.
- Micro-businesses, therefore, provide a disproportionate number of opportunities for those previously unemployed and a significant part of Scotland’s ‘demand side’.
We surveyed 580 micro-businesses and interviewed over 70 micro-business owners. Our findings included:
- Micro-businesses are a particularly diverse customer group
- There are distinctive issues around recruitment for micro-businesses
- Micro-business owners develop highly-flexible approaches to employment
- Recruitment presents micro-businesses with a range of risks and difficulties
- The level of risks perceived by micro-business owners far exceed the reality of those risks and often, in themselves, lead to decisions not to recruit – despite the fact that the business needs to recruit
- Micro-businesses often have limited capacity to manage staff.
We uncovered significant under-employment by micro-businesses and clear evidence that a support service that focused on helping micro-businesses grow through recruitment would help to tackle the risks around recruitment that micro-business owners perceived. On the basis of these findings we developed a range of detailed recommendations for national and local organisations about what an appropriate service should look like.
Outcome: Micros Untapped has proved to be highly influential report and in Scotland there are now ‘growth through recruitment’ programmes in place in over 5 local authority areas, with more in the pipeline. Glasgow’s economic development arm, ‘Jobs and Business Glasgow’ reoriented their whole strategy to focus on micro-businesses. Early results show a significant upturn in recruitment by micro-businesses in the areas where such a service has been put in place, and in two of the areas the project has been significantly expanded.
Contact Richard for more information
 Scottish Corporate Sector Statistics (2011), p.27
 Urwin, P and Buscha, F (2012). Back to Work: The role of small businesses in unemployment and enterprise. Federation of Small Businesses.
 cf 14% for Small and Medium and 9% for those employing over 250, de Kok et al, 2011
From best value to social value
Rocket Science produced a report on how public bodies could meet the objectives of the Public Services (Social Value) Act 2012. This was the culmination of an intensive consultation process with procurement leads in public bodies, voluntary and community and social enterprises and policy makers.
The report identified the challenges and made recommendations for getting the most out of this important legislation, which are just as valid today as they were a couple of years ago.
Read our report here
From Best Value to Social Value – building social value into procurement – Rocket Science report July 2012
Contact John for more information