Cristiana Orlando and Clare Hammond reflect on what makes an outcomes framework work
A fundamental must do across public and third sector organisations is to be able to understand and evidence the change you are creating. Measuring outcomes is the bread and butter of organisations. It is the area of focus of a large portion of our work here at Rocket Science.
There is a huge range of complexities to bear in mind when measuring your outcomes for example:
- Making sure you are measuring the right things – as you tend to do what you measure
- Trying to articulate impact on soft and often intangible changes you are seeing in those you work with
- Tracking long term impact so that you can report on impacts achieved after your work with an individual or community finishes.
However, the complexity we want to focus on in today’s blog is around how to coordinate impact measurement. For example, where Scotland-wide reporting is required, but activity and impact is occurring across 32 local organisations or partnerships. Scotland’s (and many other jurisdictions’) answer to this is to have a national outcomes’ framework. There are frameworks for community justice, mental health, social care etc seeking to build a comprehensive picture of impact at a national scale.
A framework has to let partners do their own thing but measure impact with consistency
The purpose of these frameworks is to enable a range of partners to do their own thing. They want to track progress, change and impact in a way that enables success to be aggregated, and in some situations compared.
An unused framework is a pointless framework
For a framework to be useful it has to be consistently applied and applicable for the vast majority of partners. More importantly a framework has to be used to be helpful.
We are all very good at creating amazing and theoretically perfect frameworks that are:
- Impossible to lower onto delivery on the ground. If it doesn’t enable local partners to tell a story that makes sense to them it is difficult to use the framework in practice.
- Not useful to those completing the data. If it isn’t useful to them then why divert already stretched staff time using it?
In our experience of reviewing and developing outcomes framework we have learned that there are several good practice elements that make a framework work:
- A good outcomes framework is concise, clear, and consistent in structure and content. Keep outcome titles short but with clear instructions on how to interpret and measure them. Keep the number of outcomes to the minimum needed. Be clear about the aim of the framework, how outcomes link together. Be open about the strengths and challenges of the framework
- Avoid obsolescence through being too vague. There is a tendency to keep outcomes very high level to ensure it is flexible and applicable for partners doing very different things. Being too vague runs the risk that the information becomes meaningless as it tries to be all things to all people.
- Create a framework that is easy to collect data. Conducting feasibility testing beforehand, ensuring the framework is co-produced involving all partners, and providing guidance on the use of proxy data are three ways to mitigate the risk of including unfeasible outcomes or indicators.
- When an outcomes framework is used across localities, it is important to balance national and local priorities. Create a clear link between national objectives and local needs by providing a breakdown of the framework’s aims at both levels. This helps partners have clarity around how they fit into the bigger picture and facilitates the collection of meaningful data locally.
- Lastly, good frameworks are closely aligned to the way business is done already. The closer the alignment the easier it is to integrate into day to day activities and greater chance of staff buying into its value
Outcomes frameworks are a powerful tool, but striking the balance between comprehensiveness, consistency and simplicity can be a tricky process.
Clare Hammond is an Associate Director and Cristiana is a consultant in our Edinburgh Office. For more information please get in touch with the Edinburgh Office on 0131 226 4949 or clare.hammond@rocketsciencelab.co.uk
