When is a distance travelled tool a bad idea?

When is a distance travelled tool a bad idea?

When is a distance travelled tool a bad idea? Clare Hammond explores when and how to use distance travelled tools

Distance travelled tools are a popular way of understanding the progression someone has made through a service. They are almost a standard part of any funder or commissioner’s monitoring ask and certainly feature heavily in evaluations and impact measurement.

But when is using a distance travelled tool a really bad idea?

The issue is that needs assessment and distance travelled tools can be seen as the same thing by funders, commissioners, service managers and others.

So, what is the difference?

  • Distance travelled tools are ways of understanding the progress an individual has made. They are particularly useful when assessing the growth in an individual’s knowledge or tracking a single outcome
  • Needs assessment tools are used by practitioners to identify needs and target interventions as part of their case management role.

Needs assessment tools are a vital part of providing holistic and person-centred support as they allow the practitioner to work through with the participant the various elements of their lives and identify the participant’s worries and needs. They tend to consider a wide variety of aspects of an individual’s life such as health, housing, relationships, employment and addiction.

It is common for practitioners to use these tools regularly throughout their engagement with a participant in order to understand the changing priorities for support.

For this reason, it can be easy to see how they could also be used to track an individual’s progression. If housing was scoring as a high area of concern and then after six weeks the concern level is significantly lower, then it could be reasonable to expect that this could be an impact of the programme.

However, needs assessment tools make terrible measures of distance travelled. They can provide a distorted and confused picture of progression for two main reasons:

  1. Progression is not a linear pathway – particularly for participants with chaotic lives – and can be distorted by how individual’s feel on a particular day. Recovery or improvement is never linear and variations in scores can be misleading when considering overall progress

 

2. Needs assessment tools can ask individuals how they feel (on a scale) on a wide range of broad issues such as employability, housing, and relationships. Practitioners quite rightly expect to see the figures on the scale to increase and decrease for reasons other than progress or regression. For example:

  • An individual may be focused on managing their addiction, so housing and relationship issues are likely to score low. Once the addiction is better managed, the focus of the individual may turn to their relationships and housing.
  • Initial scores may appear ok when individuals do not yet trust the practitioner they are working with. As the trust and relationship builds between the practitioner and participant, the individual may feel more comfortable expressing unhappiness with parts of their lives.
  • Not knowing what you don’t know can distort initial results. A participant may be happy with their housing situation initially, but as they build their self-esteem they can start to feel they deserve better. Or they can gain a better insight into  their rights when it comes to housing and they can recognise that their housing situation is unhealthy and not good enough.

In all these situations, it would be reasonable to expect to see scores worsen over time as the individual has the space to think about these areas, the trust in the practitioner to open up about what is concerning them, and the knowledge and self-esteem to know they deserve better.

There are two key differences between distance travelled tools and needs assessments to consider when working out how to measure impact:

  • Distance travelled tools should be used to test knowledge, understanding and confidence rather than feelings to avoid being distorted by a client’s feelings on a particular day
  • Distance travelled tools need to be focused and specific in what they are asking – broad questions like, ‘How are you feeling about your housing situation?’ should be reserved for needs assessment tools as they are useful questions to open up conversations about need.

So, when working out how to measure progress – beware!  What can appear to be a distance travelled tool may not provide you want you are looking for.

 

Until next time, Clare 

Clare is an Associate Director at Rocket Science who specialises in health and social care with expertise in understanding impact and conducting evaluations. To discuss anything further please get in touch at [email protected] of 0131 226 4949

Three Rocket Scientists talk about what it’s like to work at Rocket Science

Three Rocket Scientists talk about what it’s like to work at Rocket Science

Three Rocket Scientists give an insight into their day-to-day work 

Cristiana Orlando, Research Intern

Looking back to when I first started as an Research Intern in September 2018, it’s incredible how much I have learned in the span of six short months. I had just graduated from the University of Oxford with an MSc in Comparative Social Policy and I applied to the internship thinking it would be a great opportunity to dip my toes in the worlds of public policy and social justice. I can now say it’s been a lot more than that – from day one I have been working on tasks ranging from interviews with service managers and directors of health boards, to presenting to clients and writing our final reports. I have had the opportunity to work on a variety of projects including health and social care, criminal justice, and employability. During my time at Rocket Science, I have not only developed a wide set of skills, but I have also felt valued and that my work was having a genuine impact on both services and people.

Max Lohnert, Consultant

Much of what I do now as a Consultant was uncharted territory for me when I joined the Rocket Science team in Edinburgh as Research Intern in October 2017 after completing my MSc in the Psychology of Mental at the University of Edinburgh. Since then, every week has been filled with different activities for a range of projects across different sectors: ranging from doing fieldwork with vulnerable young people on employability programmes, large-scale survey analyses, all the way to conducting workshops with health service providers. Not only have I been supported through training and mentoring to develop a wide range skills, but our culturally flat structure means that there is much room for me bring my own ideas to the table and to develop my own areas of interest.

Charlotte Wu, Senior Consultant

I can honestly say that no day at Rocket Science is the same – we are always working on a revolving range of projects for a wide range of clients, which means we’re always getting to learn about new social issues and meet new people! There also isn’t a typical ‘Rocket Scientist’ – we have people from both humanities and sciences backgrounds (my BA was in English and MPhil in Gender Studies) and that variety helps us bring an interesting array of skills, interests and perspectives to any project. The thing that we all have in common is a commitment to supporting social change and fairness, and helping organisations to strengthen and demonstrate their impact.

I also appreciate that while it’s a busy and fast-paced working environment, Rocket Science is encouraging of us pursuing development (both professionally and outside the company) and work-life balance. I actually started at the organisation back in 2013 as a Consultant and decided I wanted to go back to studying, so went away and did a PhD in Global Health Humanities, freelancing for Rocket Science part-time, and then joined again full time in November 2018 as a Senior Consultant. The different ways that I’ve worked for Rocket Science over the years, which have changed with my own circumstances, is an example of the willingness to be flexible around individual staff members’ needs that I really appreciate and value.

 

See for more information about the open positions in Edinburgh or contact Clare Hammond for an informal chat on 0131 226 4949 or [email protected].

Building financial resilience in young people and families

Building financial resilience in young people and families

Rocket Science held an event on how to build finanical resilience in young people in May, find out more about the discussions, learning and access event notes and presentations.

 

Building financial resilience in people of all ages is critical to help them manage their working lives now and in the future.  Traditionally much of the support provided to help people manage their money and debt has been as a one off intervention, usually at a point of crisis.  However if people are to survive the challenges of managing their income when they are in insecure or low paid employment they are really going to need to have good financial education.

Last year Rocket Science was commissioned by four charities; Centrepoint, National Skills Academy for Financial Services, Quaker Social Action and Shelter Scotland to conduct an evaluation of their financial capability projects funded through the Money Advice Service What Works Fund.

The projects were testing new ideas and approaches to building financial capability, two of which focused on building the skills and offer for vulnerable young people and those facing transition.  One is working with families through coaching and training to develop a family approach to money management and the other has supported health care professionals in their work with people facing a combination of health and money challenges.

The event brought the learning from the evaluations, published this month, to a non-financial capability audience, including funders, commissioners and organisations working with people to improve their employment, education and life chances.

Discussions revealed that people still find talking about money or thinking ahead really hard to do.  Some of this is down to a lack of education, learnt behaviours (from parents) and in many cases being in a place where considerations about money management are just not a priority.

 

This means that people will only ask for help when they are in crisis.  Many services are designed and funded to provide one-off support, rather than as part of an on-going offer.  Often services also tend to resolve issues for people rather than giving them the skills and confidence to do it themselves.

These projects have looked at ways in which you can build better attitudes towards money management and in turn create behavioural change.

A key learning point has been the need to have an on-going trusted relationship with an advisor and recognition that people often face multiple transitions, meaning that good intentions can be derailed by life events and impact on mental health.

The main take away point for me, was that financial education needs to be recognised as a key life skill.  Importantly, we need to ensure that this becomes integral to the support we provide people of all ages to manage their personal and working lives.

The notes from the event can be found here Financing and planning for an uncertain future event note May 2018, alongside the  Presentation slides and details of the organisations who presented Information and contact details.

If you would like to find out more about our work in this area, please get in touch.

Caroline

[email protected]

 

Caroline is an Associate Director at Rocket Science based in our London office.  You can check out her profile here.

Insight from our Rocket Scientists – Working with Young People

Insight from our Rocket Scientists – Working with Young People

This week we ask some of our Rocket Scientists What is important for services that support young people?

Over the last year, we have evaluated a range of programmes supporting young people. Our clients include the Wise Group, Venture Trust, Scottish Waterways Trust, Princes Trust, and Centrepoint. This work has given us the opportunity to refresh our insights into what works when helping young people to improve their education, employability and financial resilience.

Using this insight we have asked our Rocket Scientists to share lessons they have learned about what works for young people.

Dina Papamichael, Assistant Consultant

For me, it is crucial to understand the intersecting issues which young people can face. For young people, issues around health and employment are so closely linked with self-confidence, social inclusion, substance misuse, homelessness and mental health. The most successful interventions enable a young person to overcome multiple challenges on the road to the programme’s primary objective. A programme which helps young people gain a qualification while developing a social network can have the impact of boosting self-confidence, developing work-related skills and increasing the level of stability in a young person’s life, while health services that focus on the social drivers of the sexual and mental health of its young people can obtain better and more enduring health outcomes.

Richard Scothorne, Director

The transition to work is a very tough one for many young people. Some have never really succeeded at anything at school and have often lacked the stable support of parents or other carers.  The most effective programmes we have worked with have provided different kinds of support – in other words, support that young people can chose from in terms of the quality of the relationship and their different needs from time to time.  These sources include trainers, mentors and personal coaches – someone who can provide the young person with a fixed point in a shifting world – and who will listen to them and stay with them through their journey.  In addition, working on shared tasks in small, well-managed groups – often in quite demanding conditions – can provide invaluable peer support, show people they are still appreciated even when they may be struggling, and help to build a young person’s confidence in dealing with others.

Natalie Dewison, Senior Consultant

Traditionally, the effectiveness of employability services has been assessed by looking at the number of people moving on to ‘positive destinations’ (education, employment, training or volunteering). For employability projects supporting young people facing multiple barriers to work, particularly those that only run over the course of a few weeks, this approach fails to capture the full impact of support provision.

We have found that often the most valuable outcomes are increasingly recognisable over time. Improvements in a young person’s confidence and outlook for example, which make them more resilient to knock backs and motivated by new long-term goals and ambitions. These things have the potential to greatly improve future job prospects. We are currently supporting three organisations delivering employability projects to evidence the sustainability of these outcomes. This means developing methods of longer term data collection that are simple, effective and enjoyable for the young people involved.

 

Max Lohnert, Assistant Consultant

Vulnerable young people often come from unstable backgrounds and face a range of intersecting barriers to entering employment. Considering this, our experience shows that there is no “quick fix” and that vulnerable young people benefit most from sustained engagement with employability programmes. For example, young people often benefit immensely from the trusted relationships with trainers and mentors and from the peer network they establish – and such relationships take time to build. However, since funding arrangements often require organisations to prioritise the number of young people being helped over the length of time they can engage with a single person, organisations have found other means to engage young people for longer periods of time: ranging from trainers or mentors staying in touch with a young person informally after the official completion of the programme, to building linkages and strong referral networks with other organisations along the employability pipeline.

Clare Hammond, Associate Director

The job market is becoming more complex and more competitive. Teaching young people to navigate this complexity is so important. This includes helping them look beyond the well-known and more obvious opportunities. How many jobs did you know existed when you were in high school? Doctor, teacher, engineer, nurse and banker? Probably. How about business analyst, GP practice managers, food safety consultant, project manager? Probably not.

The role that employers can play in helping young people understand the lay of the land is well evidenced. However, engagement between schools and employers often focuses around larger private sector employers. These large businesses make up a small part of employment in the UK. Most people will work for small or medium sized private firms, or the public sector. Where schools partner with small and medium sized businesses and the public sector young people are able to have a fuller understanding of the labour market.

Eleanor Sanders White, Consultant

There are many assumptions out there about young people which can interfere with our ability to reach and support them. Sometimes, the only thing in common in a group of young people is their age – we need to be careful about treating them as a homogenous group. A lack of confidence or fear of failure can often be misinterpreted as apathy. While some young people want to engage online, others need to develop a trusting relationship to engage. Many young people are technologically savvy, but some won’t have the digital skills required to engage online. While some young people will need a tailored and very supportive experience, others will interpret this as patronising. The way through this? Some of the most successful examples we have seen go straight to the source and ask the young people, others have shown a huge flexibility in how they engage with the young person to tailor for the individual walking through the door.

 

For more information on our work with young people get in touch with Clare Hammond, one of our Associate Directors [email protected] or 0131 226 4949

Prevention in Action – Invest early in housing advice according to our latest SROI on Legal Aid

Prevention in Action – Invest early in housing advice according to our latest SROI on Legal Aid

Prevention in Action – Invest early in housing advice according to our latest SROI on Legal Aid – Clare Hammond looks at the latest findings

Prevention is heralded as the long-term solution for tackling poverty and disadvantage.

Since featuring as one of the four pillars of public sector reform set out by the Christie Commission in 2011, prevention has been a firm fixture on the Scottish Government’s agenda. However, for many policy makers and service providers, it is a much spoken about ideal that still feels out of reach.

When it is done effectively prevention work can slash the costs of later support provision and reduce the demand for a wide range of pressurised services. Most importantly however, it can lessen the impacts of poverty and improve the health and wellbeing of individuals, families and their communities.

As we have previously discussed, making a decisive shift towards prevention is not without its challenges. These challenges cut across sectors and tend to centre around two areas:

  • Where funding is directed from: insufficient budget flexibility; the potential mismatch between who funds interventions and who gains from the benefits; opportunity costs associated with re-prioritising resources
  •  Evidencing return on investment: risk and uncertainty about future impacts of upfront investment; the timescales for a return on this investment

As a company, we have sought to help our clients, and others, to understand practical and realistic ways to implement prevention in an imperfect world where funding is restricted, and support provision is fragmented across different organisations and policy areas.

Don’t get us wrong, we are working with organisations across the UK to improve the coordinated efforts across policy and budget silos. However, we think that there are also a number of easy wins to implement preventative approaches that aren’t reliant on a fundamental shift in service delivery models.

The Law Society of Scotland have recently published the findings of a study they commissioned our team at Rocket Science to undertake. This study sought to identify and quantify the preventative benefits of legal aid in Scotland, focusing on housing, criminal and family law cases. We used a Social Return on Investment model for this research which measures the social, economic and financial impacts of an activity on all the relevant parties including service users, service providers and other stakeholders.

The analysis of housing cases revealed a particularly interesting finding. Many cases involved providing housing advice, often to those facing eviction as a result of debt or rent arrears. We estimated that for every £1 spent on early provision of legal advice on housing cases, there was a return of £11. This included savings to homelessness services and reductions in temporary accommodation costs, as a result of avoiding eviction and addressing growing housing arrears. It also considered the benefit to individual tenants such as improved mental and physical health and confidence in their ability to maintain their accommodation in the future.

This research provides a clear steer for local authorities, housing associations and the NHS to invest further upstream, ensuring that legal advice around housing debt is available at the earliest opportunity to avoid the higher and longer lasting costs of eviction.

Recognising the benefit of investing upstream to prevent homelessness, the Royal Borough of Kingston upon Thames is piloting a new project focused on early interventions. This project supports its service users with a wider range of issues, such as mental health, which are found to be trigger future housing problems. We are currently working with Kingston and two of their surrounding boroughs to evaluate the effectiveness of this pilot. This work will include a cost benefit analysis of their impact.

We will continue to evidence the value of investing in preventative approaches and research the most effective ways to implement new approaches. We hope that this encourages investment, so that prevention begins to feel less like a long-term ideal and more of a current opportunity.

 

Clare Hammond is an Associate Director at Rocket Science specialising in cost benefit analysis and social return on investment across a range of sectors including employability, health and social care, housing and poverty reduction. You can check out her profile here.

 

Why London’s employment initiatives need to focus on poverty reduction

Why London’s employment initiatives need to focus on poverty reduction

Does public investment into employment and skills programmes create and worsen in-work poverty?  Caroline Masundire reflects on the challenge in London and why we need to focus on poverty reduction rather than just any old job.

The challenge facing people who are unemployed or stuck in low paid employment in London is unprecedented.

Whilst we know the narrative that people should be better off in work, in London this is not always the case because of the cost and type of housing on offer, job insecurity, the quality of jobs and the benefit cap.

More than a quarter of Londoners live in poverty (27%) and the majority of people in poverty are in working households. This comes from my ‘go to’ resource of choice – the London Poverty Profile*, which also highlights the differences in each of the London boroughs.

A recent find on twitter highlighted how worse off people in low pay are in London. They are more likely than anywhere else in the country to have experienced the greatest rise in living costs, yet the lowest growth in wages, well behind those in Scotland, the East and South East of England.

And there are differences in how this plays out in different parts of the capital.

According to the London Poverty Profile*, there are five outer London boroughs reported to have the highest proportion of jobs that are below the London Living Wage (£9.75 phr). Landlord possession rates (figures for London are more than double national rates) are also higher in outer London and there are more people in poverty who are renting privately than in social housing.

These statistics chime with the work we have been doing in several London boroughs over the past 12 months to help better understand local needs and design appropriate solutions.  

Many people are living in precarious circumstances directly linked to reduced income and ability to afford housing:

  

  • for those in work
  • those needing to find work because of welfare reform changes
  • and those that cycle in and out of work because they cannot sustain employment.

Our experience of reviewing local employment initiatives is that whilst they are rightly focused on helping people get a job, there can often be little evidence or tracking to measure the impact of their intervention in helping a person sustain that job and improve their financial situation.

The quality and pay level of a job should be equally important as getting a job in the first place. 

There needs to be a shift in focus from getting a job, to getting into a London Living Wage job, as there is a danger that investments being made to help people get back to work, could be making in-work poverty worse.

We also think that employment interventions used to help manage service demand, or support early intervention, might not lead to the savings for services such as housing and social care that councils expect.

Rocket Science has developed a model and cost benefit analysis that helps to determine council savings linked to poverty reduction. This information makes a compelling case for councils and local partners to focus their employment interventions on securing London Living Wage jobs, or help those in lower-paid jobs to access better paid employment.

By shifting employment interventions to focus on poverty reduction, this brings sustained returns for the person in terms of increased income, as well as tangible savings for the council.

 

Caroline is an Associate Director at Rocket Science based in our London office.  You can check out her profile here.