When is a distance travelled tool a bad idea? Clare Hammond explores when and how to use distance travelled tools
Distance travelled tools are a popular way of understanding the progression someone has made through a service. They are almost a standard part of any funder or commissioner’s monitoring ask and certainly feature heavily in evaluations and impact measurement.
But when is using a distance travelled tool a really bad idea?
The issue is that needs assessment and distance travelled tools can be seen as the same thing by funders, commissioners, service managers and others.
So, what is the difference?
- Distance travelled tools are ways of understanding the progress an individual has made. They are particularly useful when assessing the growth in an individual’s knowledge or tracking a single outcome
- Needs assessment tools are used by practitioners to identify needs and target interventions as part of their case management role.
Needs assessment tools are a vital part of providing holistic and person-centred support as they allow the practitioner to work through with the participant the various elements of their lives and identify the participant’s worries and needs. They tend to consider a wide variety of aspects of an individual’s life such as health, housing, relationships, employment and addiction.
It is common for practitioners to use these tools regularly throughout their engagement with a participant in order to understand the changing priorities for support.
For this reason, it can be easy to see how they could also be used to track an individual’s progression. If housing was scoring as a high area of concern and then after six weeks the concern level is significantly lower, then it could be reasonable to expect that this could be an impact of the programme.
However, needs assessment tools make terrible measures of distance travelled. They can provide a distorted and confused picture of progression for two main reasons:
- Progression is not a linear pathway – particularly for participants with chaotic lives – and can be distorted by how individual’s feel on a particular day. Recovery or improvement is never linear and variations in scores can be misleading when considering overall progress
- Needs assessment tools can ask individuals how they feel (on a scale) on a wide range of broad issues such as employability, housing, and relationships. Practitioners quite rightly expect to see the figures on the scale to increase and decrease for reasons other than progress or regression.
For example:
- An individual may be focused on managing their addiction, so housing and relationship issues are likely to score low. Once the addiction is better managed, the focus of the individual may turn to their relationships and housing.
- Initial scores may appear ok when individuals do not yet trust the practitioner they are working with. As the trust and relationship builds between the practitioner and participant, the individual may feel more comfortable expressing unhappiness with parts of their lives.
- Not knowing what you don’t know can distort initial results. A participant may be happy with their housing situation initially, but as they build their self-esteem they can start to feel they deserve better. Or they can gain a better insight into their rights when it comes to housing and they can recognise that their housing situation is unhealthy and not good enough.
In all these situations, it would be reasonable to expect to see scores worsen over time as the individual has the space to think about these areas, the trust in the practitioner to open up about what is concerning them, and the knowledge and self-esteem to know they deserve better.
There are two key differences between distance travelled tools and needs assessments to consider when working out how to measure impact:
- Distance travelled tools should be used to test knowledge, understanding and confidence rather than feelings to avoid being distorted by a client’s feelings on a particular day
- Distance travelled tools need to be focused and specific in what they are asking – broad questions like, ‘How are you feeling about your housing situation?’ should be reserved for needs assessment tools as they are useful questions to open up conversations about need.
So, when working out how to measure progress – beware! What can appear to be a distance travelled tool may not provide you want you are looking for.
Until next time, Clare
Clare is an Associate Director at Rocket Science who specialises in health and social care with expertise in understanding impact and conducting evaluations. To discuss anything further please get in touch at clare.hammond@rocketsciencelab.co.uk of 0131 226 4949
