The impact of the mental health crisis on evaluations

Last week was Mental Health Awareness Week.  Considering where we were in terms of thinking and talking openly about mental health just a few years ago, we have come on leaps and bounds as a society in supporting family, friends, and colleagues to better manage their mental health. 

But we all know the struggles that people are facing to access professional help. The prevalence of poor mental health is growing as shown in this ONS report and is affecting the economy with a 22% increase in economic inactivity in the labour market, and now 1 in 4  (rising from 1 in 10 in 2017) young people reporting a probable mental disorder.

Latest research from Healthwatch, highlights the hidden waiting lists for a wide range of NHS treatment, with one in three patients surveyed unable to secure a referral. Delays and repeat requests for referrals are creating stress and anxiety.  In October last year the Royal Society of Psychiatrists identified that 43% of respondents to a survey said that that the wait between initial referral and when treatment begins has made their mental health worse, with over three quarters of respondents on these ‘hidden’ waiting lists resorting to seeking help from emergency services or a crisis line.  

The challenge for services that are providing support, advice and help for people and families is that the lack of mental health provision is materially impacting on the progress and outcomes these services are delivering Whether its a financial resilience, employment, wellbeing or navigator service we are hearing the same issues.

From an evaluator’s perspective we think the following five steps will help organisations and funders better understand the impact of these challenges on services, how to manage expectations and better adapt evaluation questions and methods. 

1. Recognise the problem. We know that this sounds obvious, but in our experience whilst the issues of not accessing mental health support are known, the impact of this on the model of delivery is often not understood.  We are working in complex and difficult times and making staff, trustees, stakeholders and funders aware of the impact on delivery will bring greater recognition and acceptance of the issues.
2. Understanding the impact on staff mental health. This also seems obvious, but we are seeing a growing trend of what we are highlighting as ‘Vicarious Anxiety’ not as severe as is seen in cases of Vicarious Trauma.  However, because of the difficulties faced by frontline staff in managing more complex cases and the worsening of peoples’ mental health conditions while they cannot get help, the mental health of staff is affected.  This is particularly true where staff feel helpless, become worried about targets or lack of progress, and do not have the skills or competence to deal with this increasing complexity.  This requires greater support, help and training for staff particularly in knowing their limits of competence, helping them to be more resilient and to have the psychological safety in place to be able to tell their peers and line manager when they need support.
3. Review the fidelity of the service model. We recommend reviewing the service model and to see where there are elements of divergence around delivery.  In our experience this begins and ends with the referral and triage processes at the beginning of a client’s journey, ensuring that you are attracting the right level of clients to the service. Often targets can take over and if referrals are low then widening the criteria for support to encourage more referrals will affect the fidelity of the model. If it is clear that the model needs to adapt, then track the changes and understand what would need to shift around any Theory of Change/Logic model that is underpinning the evaluation.  
4. Review the evaluation questions and methodology. Following a review of the model you then need to reflect on the evaluation process and whether the questions and methods you are using are capturing what is happening.  In our experience this reflection can come too late and therefore materially impact on the evaluation findings and analysis.  We think the use of case studies to reflect the challenges people are facing, understanding the prevalence and scale of mental health challenges (which requires assessment and data) and regular reflections with staff and referral agencies is critical to understanding what has shifted and how this will impact the evaluation. This is particularly important when using a trauma-informed approach to the research. Developing a revised method and approach supported by a rationale for change will be needed.
5. Tell the funder, commissioner, senior management. Finally, tell people about the challenges you are facing and prepare them about what to expect.  Funders are generally flexible and willing to adapt if things have changed and are open to understanding intersectional issues recognising the external environment is constantly changing.  Being open will help manage expectations, and although you may need to negotiate about what was expected, this will mitigate any challenge further on in the future.

Services do not operate in a vacuum and neither do evaluations.  Things change, circumstances change, Covid 19 taught us this well.  These challenges are not likely to go away and are more likely to increase as access to mental health support continues to be a cause of frustration, anxiety and worsening conditions for many.

Caroline Masundire is Joint Managing Director and Co-owner of Rocket Science.