When is a distance travelled tool a bad idea?

When is a distance travelled tool a bad idea?

When is a distance travelled tool a bad idea? Clare Hammond explores when and how to use distance travelled tools

Distance travelled tools are a popular way of understanding the progression someone has made through a service. They are almost a standard part of any funder or commissioner’s monitoring ask and certainly feature heavily in evaluations and impact measurement.

But when is using a distance travelled tool a really bad idea?

The issue is that needs assessment and distance travelled tools can be seen as the same thing by funders, commissioners, service managers and others.

So, what is the difference?

  • Distance travelled tools are ways of understanding the progress an individual has made. They are particularly useful when assessing the growth in an individual’s knowledge or tracking a single outcome
  • Needs assessment tools are used by practitioners to identify needs and target interventions as part of their case management role.

Needs assessment tools are a vital part of providing holistic and person-centred support as they allow the practitioner to work through with the participant the various elements of their lives and identify the participant’s worries and needs. They tend to consider a wide variety of aspects of an individual’s life such as health, housing, relationships, employment and addiction.

It is common for practitioners to use these tools regularly throughout their engagement with a participant in order to understand the changing priorities for support.

For this reason, it can be easy to see how they could also be used to track an individual’s progression. If housing was scoring as a high area of concern and then after six weeks the concern level is significantly lower, then it could be reasonable to expect that this could be an impact of the programme.

However, needs assessment tools make terrible measures of distance travelled. They can provide a distorted and confused picture of progression for two main reasons:

  1. Progression is not a linear pathway – particularly for participants with chaotic lives – and can be distorted by how individual’s feel on a particular day. Recovery or improvement is never linear and variations in scores can be misleading when considering overall progress

 

2. Needs assessment tools can ask individuals how they feel (on a scale) on a wide range of broad issues such as employability, housing, and relationships. Practitioners quite rightly expect to see the figures on the scale to increase and decrease for reasons other than progress or regression. For example:

  • An individual may be focused on managing their addiction, so housing and relationship issues are likely to score low. Once the addiction is better managed, the focus of the individual may turn to their relationships and housing.
  • Initial scores may appear ok when individuals do not yet trust the practitioner they are working with. As the trust and relationship builds between the practitioner and participant, the individual may feel more comfortable expressing unhappiness with parts of their lives.
  • Not knowing what you don’t know can distort initial results. A participant may be happy with their housing situation initially, but as they build their self-esteem they can start to feel they deserve better. Or they can gain a better insight into  their rights when it comes to housing and they can recognise that their housing situation is unhealthy and not good enough.

In all these situations, it would be reasonable to expect to see scores worsen over time as the individual has the space to think about these areas, the trust in the practitioner to open up about what is concerning them, and the knowledge and self-esteem to know they deserve better.

There are two key differences between distance travelled tools and needs assessments to consider when working out how to measure impact:

  • Distance travelled tools should be used to test knowledge, understanding and confidence rather than feelings to avoid being distorted by a client’s feelings on a particular day
  • Distance travelled tools need to be focused and specific in what they are asking – broad questions like, ‘How are you feeling about your housing situation?’ should be reserved for needs assessment tools as they are useful questions to open up conversations about need.

So, when working out how to measure progress – beware!  What can appear to be a distance travelled tool may not provide you want you are looking for.

 

Until next time, Clare 

Clare is an Associate Director at Rocket Science who specialises in health and social care with expertise in understanding impact and conducting evaluations. To discuss anything further please get in touch at [email protected] of 0131 226 4949

Three Rocket Scientists talk about what it’s like to work at Rocket Science

Three Rocket Scientists talk about what it’s like to work at Rocket Science

Three Rocket Scientists give an insight into their day-to-day work 

Cristiana Orlando, Research Intern

Looking back to when I first started as an Research Intern in September 2018, it’s incredible how much I have learned in the span of six short months. I had just graduated from the University of Oxford with an MSc in Comparative Social Policy and I applied to the internship thinking it would be a great opportunity to dip my toes in the worlds of public policy and social justice. I can now say it’s been a lot more than that – from day one I have been working on tasks ranging from interviews with service managers and directors of health boards, to presenting to clients and writing our final reports. I have had the opportunity to work on a variety of projects including health and social care, criminal justice, and employability. During my time at Rocket Science, I have not only developed a wide set of skills, but I have also felt valued and that my work was having a genuine impact on both services and people.

Max Lohnert, Consultant

Much of what I do now as a Consultant was uncharted territory for me when I joined the Rocket Science team in Edinburgh as Research Intern in October 2017 after completing my MSc in the Psychology of Mental at the University of Edinburgh. Since then, every week has been filled with different activities for a range of projects across different sectors: ranging from doing fieldwork with vulnerable young people on employability programmes, large-scale survey analyses, all the way to conducting workshops with health service providers. Not only have I been supported through training and mentoring to develop a wide range skills, but our culturally flat structure means that there is much room for me bring my own ideas to the table and to develop my own areas of interest.

Charlotte Wu, Senior Consultant

I can honestly say that no day at Rocket Science is the same – we are always working on a revolving range of projects for a wide range of clients, which means we’re always getting to learn about new social issues and meet new people! There also isn’t a typical ‘Rocket Scientist’ – we have people from both humanities and sciences backgrounds (my BA was in English and MPhil in Gender Studies) and that variety helps us bring an interesting array of skills, interests and perspectives to any project. The thing that we all have in common is a commitment to supporting social change and fairness, and helping organisations to strengthen and demonstrate their impact.

I also appreciate that while it’s a busy and fast-paced working environment, Rocket Science is encouraging of us pursuing development (both professionally and outside the company) and work-life balance. I actually started at the organisation back in 2013 as a Consultant and decided I wanted to go back to studying, so went away and did a PhD in Global Health Humanities, freelancing for Rocket Science part-time, and then joined again full time in November 2018 as a Senior Consultant. The different ways that I’ve worked for Rocket Science over the years, which have changed with my own circumstances, is an example of the willingness to be flexible around individual staff members’ needs that I really appreciate and value.

 

See for more information about the open positions in Edinburgh or contact Clare Hammond for an informal chat on 0131 226 4949 or [email protected].

Insight from our Rocket Scientists – Working with Young People

Insight from our Rocket Scientists – Working with Young People

This week we ask some of our Rocket Scientists What is important for services that support young people?

Over the last year, we have evaluated a range of programmes supporting young people. Our clients include the Wise Group, Venture Trust, Scottish Waterways Trust, Princes Trust, and Centrepoint. This work has given us the opportunity to refresh our insights into what works when helping young people to improve their education, employability and financial resilience.

Using this insight we have asked our Rocket Scientists to share lessons they have learned about what works for young people.

Dina Papamichael, Assistant Consultant

For me, it is crucial to understand the intersecting issues which young people can face. For young people, issues around health and employment are so closely linked with self-confidence, social inclusion, substance misuse, homelessness and mental health. The most successful interventions enable a young person to overcome multiple challenges on the road to the programme’s primary objective. A programme which helps young people gain a qualification while developing a social network can have the impact of boosting self-confidence, developing work-related skills and increasing the level of stability in a young person’s life, while health services that focus on the social drivers of the sexual and mental health of its young people can obtain better and more enduring health outcomes.

Richard Scothorne, Director

The transition to work is a very tough one for many young people. Some have never really succeeded at anything at school and have often lacked the stable support of parents or other carers.  The most effective programmes we have worked with have provided different kinds of support – in other words, support that young people can chose from in terms of the quality of the relationship and their different needs from time to time.  These sources include trainers, mentors and personal coaches – someone who can provide the young person with a fixed point in a shifting world – and who will listen to them and stay with them through their journey.  In addition, working on shared tasks in small, well-managed groups – often in quite demanding conditions – can provide invaluable peer support, show people they are still appreciated even when they may be struggling, and help to build a young person’s confidence in dealing with others.

Natalie Dewison, Senior Consultant

Traditionally, the effectiveness of employability services has been assessed by looking at the number of people moving on to ‘positive destinations’ (education, employment, training or volunteering). For employability projects supporting young people facing multiple barriers to work, particularly those that only run over the course of a few weeks, this approach fails to capture the full impact of support provision.

We have found that often the most valuable outcomes are increasingly recognisable over time. Improvements in a young person’s confidence and outlook for example, which make them more resilient to knock backs and motivated by new long-term goals and ambitions. These things have the potential to greatly improve future job prospects. We are currently supporting three organisations delivering employability projects to evidence the sustainability of these outcomes. This means developing methods of longer term data collection that are simple, effective and enjoyable for the young people involved.

 

Max Lohnert, Assistant Consultant

Vulnerable young people often come from unstable backgrounds and face a range of intersecting barriers to entering employment. Considering this, our experience shows that there is no “quick fix” and that vulnerable young people benefit most from sustained engagement with employability programmes. For example, young people often benefit immensely from the trusted relationships with trainers and mentors and from the peer network they establish – and such relationships take time to build. However, since funding arrangements often require organisations to prioritise the number of young people being helped over the length of time they can engage with a single person, organisations have found other means to engage young people for longer periods of time: ranging from trainers or mentors staying in touch with a young person informally after the official completion of the programme, to building linkages and strong referral networks with other organisations along the employability pipeline.

Clare Hammond, Associate Director

The job market is becoming more complex and more competitive. Teaching young people to navigate this complexity is so important. This includes helping them look beyond the well-known and more obvious opportunities. How many jobs did you know existed when you were in high school? Doctor, teacher, engineer, nurse and banker? Probably. How about business analyst, GP practice managers, food safety consultant, project manager? Probably not.

The role that employers can play in helping young people understand the lay of the land is well evidenced. However, engagement between schools and employers often focuses around larger private sector employers. These large businesses make up a small part of employment in the UK. Most people will work for small or medium sized private firms, or the public sector. Where schools partner with small and medium sized businesses and the public sector young people are able to have a fuller understanding of the labour market.

Eleanor Sanders White, Consultant

There are many assumptions out there about young people which can interfere with our ability to reach and support them. Sometimes, the only thing in common in a group of young people is their age – we need to be careful about treating them as a homogenous group. A lack of confidence or fear of failure can often be misinterpreted as apathy. While some young people want to engage online, others need to develop a trusting relationship to engage. Many young people are technologically savvy, but some won’t have the digital skills required to engage online. While some young people will need a tailored and very supportive experience, others will interpret this as patronising. The way through this? Some of the most successful examples we have seen go straight to the source and ask the young people, others have shown a huge flexibility in how they engage with the young person to tailor for the individual walking through the door.

 

For more information on our work with young people get in touch with Clare Hammond, one of our Associate Directors [email protected] or 0131 226 4949

How does a health and social care organisation become a big player in employment? The power of stepping back from a diagnosis.

How does a health and social care organisation become a big player in employment? The power of stepping back from a diagnosis.

Clare Hammond looks at why health, social care and employment are so important to each other

18 months ago, we were working with a Scottish Health and Social Care Partnership (HSCP) and were struck by what we found. This HSCP was spending the equivalent of 50% of the value of the new national programme for those further from work, on helping their service users into employment. How does a HSCP end up being such a big player in employment?

More and more, services are stepping back from a diagnosis, or presenting issue, and looking at the whole person. A perfect illustration of buzz words such as ‘person centred’ and ‘holistic’. In health and social care, this is driven by a growing understanding that medical conditions are often driven by, and in turn drive, other issues such as trauma, poverty, social inclusion, and worklessness.

These intersections are what has driven the growth in health and social care integration, information and support services, complementary therapies, and social prescribing, and where prevention has found a practical expression.

Subsequently, services have needed to work across their typical service boundaries. These partnerships find two main forms (1) strategic and structural partnerships, and (2) practical and operational partnerships at the coal face.

Much of the focus on health and social care integration in both England and Scotland has focused on structural integration. However, we have found numerous examples of long standing, pragmatic and effective operational partnerships as front-line staff recognise the need to engage with other services to get the best outcome for their clients.

So, while strategic partnerships often feel that they a have responsibility to support partnership working across every level, there is a lot that strategic partners can learn from front line delivery staff for whom partnership working has long been part of their everyday approach to support provision.

Coming back to the HSCP mentioned at the beginning of this blog, how did they end up here? This HSCP had 16 employability related projects covering a variety of health and social care services. Despite operating independently over a period of more than 5 years, front-line staff working across all projects had identified the important role that employment (or the prospect of employment) could play in managing an individual’s health and wellbeing and maintaining their progression. As such, working towards employment became part of a recovery journey in mental health, addictions and criminal justice, and a way of improving health and care outcomes for those with disabilities, or young people raised in care.

From an employability perspective, these service users were effectively pre-stage 1 in the Strategic Skills Pipeline. By encouraging participants to lift their aspirations of future employment and begin setting goals. these HSCP services effectively prepare them for mainstream employment services.

This approach is echoed by new national employability programmes (Work and Health Programme and Fair Start Scotland) being developed across the UK which have a particular focus on supporting those with multiple barriers to employment. The only way these programmes are going to be successful is if they link in with projects like those being delivered through the HSCP and encourage people to think about employment as part of their recovery journey. Doing so should improve the level of engagement when they enter the first stage of the employability pipeline and the outcomes achieved.

To reach this point, health and social care services can learn from this HSCP, step back from the diagnosis and consider how they can help service users make the next steps on their longer term pathway to employment.

Clare Hammond is an Associate Director at Rocket Science specialising in health and social care including evaluation, strategy and organisational development, cost modelling and cost benefit analysis. You can check out her profile here.

This blog is part of a wider series on Health and Social Care Integration. Other blogs in this series can be found here

Health and Social Care – What’s the point?

Health and Social Care – What’s the point?

Health and Social Care Integration – What’s the point? Clare Hammond explores the early evidence on the impact of health and social integration

Health and Social Care integration has been in the spotlight in both England and Scotland with promises of large financial savings coming from both countries. But can integration really be the silver bullet for these financially stretched services?

The Scottish Government estimated a potential annual savings of between £138 million to £157 million from the integration of Health and Social Care. This represents less than 2% of the more than £8 billion of spend integrated bodies are responsible for. When compared with the expected increase in demand of between 18 and 29% between 2010 and 2030 it is hard to see integration as the saviour of financial woes.

A core element of integration in England is the Better Care Fund which was estimated to save £511 million in its first year, a little under 10% of the pooled budget of £5.3 billion. Interestingly, the NHS Sustainability and Transformation Fund for 2016/17 had £1.8 billion of its £2.1 billion rediverted from integration to fund NHS deficits, indicating the scale of the financial situation.

So, it seems unrealistic to expect health and social care to plug the growing gap between demand and public budgets. But how is integration going in achieving the savings that are expected?

The primary assumption for integration related savings is that a more preventative and social care focused approach can steer people away from the more expensive emergency hospital-based care.

The National Audit Office (NAO) concluded that integration in England has “yet to show that integration leads to better outcomes for patients”. NHS England had hoped to reduce hospital activity growth from 2.9% to 1.3% (by 2020/2021), however both the number of admissions and delayed discharge days increased in 2015/16 at a total extra cost of £457 million. Audit Scotland are due to report on the progress in Scotland in November of this year.

Of course, you can’t know what would have occurred without a counterfactual. The NAO themselves acknowledge that there is a lack of comparable data across different care settings to place evaluation findings into context.  So, what does the evidence have to say about integration’s ability to reduce hospital admissions?

A University of Oxford study published in 2017 concluded that, while the policy discourse around Health and Social Care has pushed the approach as a solution for reducing emergency admissions, the evidence available tells a very different story. According to them, there is a mixed view on the success of a number of integrated care programmes that have been evaluated so far. Of 13 reviews conducted, only three had a significant impact on the number of hospitalisations that occurred. This review included studies on Case Management systems (a common component of health and social care integrated models) found that this approach did not have an impact on hospital admissions.

 

The Kings Fund has reported on a range of examples of integration at the coal face – where integration of services occurs rather than the integration of the institutions that sit behind them. One example described is in east Kent where professionals are working together across five community hubs coordinating the care individuals identified as high risk of hospitalisation. These community hubs include teams of “GPs, community nurses, social care workers, mental health professionals, pharmacists, health and social care co-ordinators and others”. They conclude that early findings suggest “that these changes have led to year-on-year reductions in emergency admissions to hospitals”.

So why is the evidence available so mixed? Well, perhaps it is as the Barker Commission concluded in 2014 of integration in England. “Moving to a single budget with a single commissioner is not a sufficient condition to tackle the myriad problems of integration that face health and social care. But we believe it is a necessary one.” Or as Scottish Government said “[s]tructural integration brings few benefits unless accompanied by many other changes”.

There may also be a problem with the focus on structural change rather than on the behavioural and relationships changes that are needed to drive real transformation.  Research by the King’s Fund identified that there was “a strong recognition that the relational and behavioural aspects of transformation deserve as much, if not more, attention than technical and structural aspects.” So integration bodies may be looking in the wrong place and not paying enough attention to how staff behave and work together on the ground

Is it too soon to tell? While Health and Social Care Integration has been discussed for more than a decade, large scale action is still relatively new. In addition, integration is occurring in already financially strained and very complex environments. It is hard to isolate the various strands in this complexity to understand what has worked, what hasn’t, and why.

Indeed, a frequently quoted example of ‘best practice’ is Greater Manchester who are operating multi-disciplinary neighbourhood teams, intensive support targeted at those considered high risk, and driving the use of community and primary health care settings to reduce hospital use. But when you explore what makes Manchester work so well it appears to boil down to two factors:

  • Their long established public-sector partnership that crosses typical service boundaries.
  • Their ability to access additional funding for integration activities from the Sustainability and Transformation Plans Fund that reduces the choice between either funding rising financial pressures or integration initiatives.

 

Clare Hammond is an Associate Director at Rocket Science specialising in health and social care including evaluation, strategy and organisational development, cost modelling and cost benefit analysis.You can check out her profile here.

This blog is part of a wider series on Health and Social Care Integration. Other blogs in this series can be found here

.