chevron icon twitter logo facebook logo linkedin logo Youtube logo download icon quote icon posted icon clock icon author icon arrow icon arrow icon plus icon Search icon location icon location icon document icon plus-alt

Setting and measuring outcomes

A guide to identifying, defining and measuring outcomes for social impact bonds

Download as PDF
Outcomes measurement

This guide will walk you through setting and measuring outcomes. While it covers what you need to consider if you are developing an outcome based contract or social impact bond, the content is relevant for anyone who is trying to work out how to set and measure outcomes for any purpose. It will explain what an outcomes framework is, how to identify and measure outcomes with examples, and explain key concepts

This is the second edition of this guide. We would like to thank Neil Stanworth from ATQ Consultants who wrote the first edition of this guide. He shared considerable expertise and experience with us and we were able to build on this for our updated version.

We would also like to thank Roger Bullen from Triodos Bank and Jen Byrne from Future Public for reviewing and commenting on the drafts

An overview

By definition, setting the right outcomes, and measuring them effectively, is one of the most important aspects of an outcomes-based contract. The achievement of outcomes will determine how much is paid by the commissioners to the providers or investors. Also, the definition of outcomes directly affects whether commissioners achieve value for money from the contract. 

This guidance is about setting and measuring outcomes. If you are putting together an outcomes-based contract, we would advise you to also read the Payment Mechanism guide (to be published December 2018). And regardless of what sort of contract you are letting, we would advise you to read An Introduction to Evaluation. As outcomes inform payments to be made, and the impact of the outcomes need to be measured and evaluated, these three guides are inextricably linked. We will refer to these guides and other resources throughout. 

Furthermore, our projects database shows all the SIBs that have been launched and identifies their outcomes. You can search different policy areas so you will be able to find a SIB that is similar to one that you are developing. 

For more in depth information you can look at our case studies. These focus on a smaller number of SIBs in greater detail and are a great resource to look at to see what others have done.    

Chapter 1

Why measure outcomes?

2 mins read

Whilst setting and measuring outcomes is crucial for determining the payments made, this is not the only thing it is needed for. Even conventional service-based contracts and grants can be outcomes-focused when you set outcomes upfront and measure the achievement of them. Before working out how to set and measure outcomes, it is worth pausing to articulate your objective in doing so. There could be many reasons, but ultimately, we think they boil down to three.

1. To manage performance or learn how to get better

Managing performance and learning how to get better are two sides of the same coin, partly depending on whether you are a commissioner or provider, as well as the type of relationship you set up. 

The overarching point is that one very good reason to set and measure outcomes is formative. You will use the information to indicate whether a project is moving closer to its ultimate objective or not. You will make decision based on the information. For example, you might offer some advice to a provider, or offer your team a certain type of training.

Setting and measuring outcomes gives you more information to ensure you are making the right decisions that will move you towards your ultimate objective. It's not enough information on its own as you will want qualitative information as well, to add colour to the numbers. A good combination is best.

2. To evaluate whether something works (but not why it works – or doesn’t) 

Measuring outcomes is a very important part of the process of evaluation, which we cover in a separate guide, see An Introduction to Evaluation. Some evaluations can be formative as they help with learning how to do better. However, this point refers to the summative evaluation which comes at the end and gives you an idea of how effective it was. 

An impact evaluation is the most robust and uses quantitative methods (statistics) to show whether a project has or hasn't produced better outcomes than what might have happened anyway. Qualitative evaluation (discussions, interviews, stories) is also needed to understand why something worked or didn't. Together, this information can help you and others to decide whether to continue a project, change it, or stop it altogether. 

3. To provide a means for payment

Outcome based contracts have seen increasing use in the UK in recent years, particularly through social impact bonds. These are where partial or full payment is made between a government agency and an external provider (or sometimes, between different levels of government) based on a set of pre-agreed outcomes. 

The idea is that by paying for outcomes, rather than specifying particular activities (which is a more typical method of contracting), the commissioner (or 'payer') gives the provider more flexibility to deliver the service in a way that will work best, learning as they go along. 

Proponents also argue it gives providers a greater incentive for success, and reduces the financial risk to government if a project fails. To pay for outcomes, you need to be extra thoughtful about how you set and measure them. After reading this guide, you may wish to read the Payment Mechanism guide.

Once you have an understanding as to why you are measuring outcomes, you can then begin developing an outcomes framework. 

Professor Mildred Warner from Cornell University talks about the importance of measuring outcomes not outputs

Chapter 2

What is an outcomes framework?

3 mins read

One of the potential benefits of SIBs is that it often give providers and investors greater freedom to operate than contracts that pay on outputs rather than outcomes. With no prescribed service delivery method, commissioners need to create a robust outcomes framework to ensure that the service delivered is acceptable to all stakeholders and there is an acceptable means of achieving outcomes. 

A robust outcomes framework sets the groundwork for your SIB project. It needs to define the following: 

  • the outcomes to be used
  • the measures to be applied to each outcome, which show whether an outcome has been achieved or not
  • the specific targets to be applied to each measure, that determine the level of achievement at which outcome payments will be made
  • when measurement takes place

When developing this framework, it is useful to refer to the contract triangle. This resource was developed to set out the key components of developing an outcomes contract. It explains that an outcomes contract needs to be robust on three things: the cohort of beneficiaries that your SIB project will support, the outcomes that the SIB aims to achieve, and the price that will be paid for the outcomes. This technical guide will focus on the second point in order to support you to come up with outcomes that align to the policy objectives.  

Designing a robust outcomes framework
Figure 1: contract triangle for designing a robust framework

Outcomes, measures and targets

Before developing your outcomes framework, it is important to understand the key concepts that you will be working with. Many of those involved in setting outcomes, and much of the guidance around, tends to use the terms outcomemeasure and target loosely and interchangeably. You may question the difference between them. However, they do mean slightly different things, and should really be seen as separate parts of the process of developing an outcomes framework and associated contract:
  • outcome is what changes for an individual as the result of a service or intervention e.g. improved learning in school, better mental health, etc.
  • outcome measure (also termed an indicator) is the specific way the commissioner chooses to determine whether that outcome has been achieved, e.g. a test score. Often this encompasses a single dimension of an outcome.
  • outcome target (also termed metric or trigger) is the specific value attached to the measure for the purposes of determining whether satisfactory performance has been achieved. e.g. a test score of 95 out of 100 or improvement of 30 points in a test score over a 5 month period. Under a PbR contract or SIB, these metrics will usually determine whether a payment is made to the provider or investor.

Both measures and targets may relate to what is achieved by and for an individual within the cohort, or to performance across the cohort as a whole.  

It is helpful to think about outcomes and measures separately when considering the framework as a whole. It may be possible to define outcomes that fully relate to the policy objective, but measures may not be comprehensive in capturing all facets of an outcome. The fact that an outcome is difficult to measure does not make the outcome invalid, but it may mean it is not appropriate to attach a payment to it (as you might in an outcome based contract). 

It is also advisable to have a fairly settled view of the outcomes and measures to be applied to a contract before starting to assign specific values to them as metrics or triggers.

Examples of outcomes, measures and targets

Youth unemployment reduction
Example 1: Youth unemployment reduction
Improved emotional wellbeing of young people
Example 2: Improving emotional wellbeing of young people
Managing long term conditions
Example 3: Managing long term conditions

Chapter 3

Identifying the right outcomes

7 mins read

With an understanding of the key concepts you will need to start identifying the right outcomes for your project. The chapter will start with a checklist for identifying the right outcomes. It will them go into more detail about the different types of outcomes there are so you can consider what might be appropriate for your project. 

It is important to note that there are no hard and fast rules about what constitutes the “right” set of outcomes and measures for a particular project or contract. Project teams should expect that the process of developing outcomes will be both iterative and progressive.

Iterative– It is likely that developing an outcomes framework many outcomes will be discarded and there is an element of trial and error. For example, if there was challenges to do with data collection, the outcome may not work. 

Progressive – In the early stages outcomes don’t need to specified and defined in great detail, but as work progresses they will need a closer definition. Ultimately, they will need to be clearly and unambiguously defined in the contract.  

Different stakeholders may have different perspectives on what constitutes a robust and viable outcome measure and it is crucial to negotiate as you go along. In most cases, the task of setting outcomes is likely to proceed incrementally as you develop the business case. It will be informed and influenced by other decisions, particularly designing the payment mechanism for your outcomes (if designing an outcomes contract).

Checklist for identifying outcomes

You will need to ask yourself whether you are able to answer the following questions below when identifying outcomes. 

  • Can you narrow down to one primary outcome and several secondary outcomes? 
  • Do the outcomes align to the policy objectives of the project? This includes the social problem that the contract aims to address, as well as the financial benefits it will bring.
  • What hard or soft outcomes will you use?
  • Do you need proxy outcomes if your outcomes are difficult to measure directly? 
  • Do you need to set outcomes that show progression or will a binary yes/no be enough?  
  • Are the outcomes achievable through a social intervention?
  • Are they acceptable to all stakeholders? 
  • Do they reflect the priority of the service users? Has this been tested?  

Primary and secondary outcomes

When you are selecting the correct outcomes you will need to pin down the primary outcome and the other important secondary outcomes.  

primary outcome – this is the single most important outcome from the contract, and the one that the commissioner most wants to see positively impacted. It is likely to reflect the most important outcome for the commissioner in policy terms and/or delivers the greater financial benefit to the commissioner. 

secondary outcome - these are the other important outcomes that the commissioner wishes to see improved. They may capture a different dimension of the programme intent and help to reinforce and sustain the primary outcome. They may capture stakeholder interests that are not reflected in the primary outcome and offer further benefits to the commissioners. They may also counterbalance perverse incentives.

Limiting and classifying the number of outcomes is simpler to manage, as having multiple outcomes make it harder to predict when they will occur. In an outcome based contract, including multiple payment outcomes makes it harder for commissioners to forecast expenditure, and harder for providers (and investors) to predict income and cashflow. 

At this point it will be useful to refer to the payment mechanism guide to understand how outcome measures are linked to payment.

In some contracts, the commissioner may wish to measure a larger number of outcomes than the norm. In these cases, we recommend defining and specifying outcomes that will be measured but will not be linked to payment. Commissioners should however take account of the cost and effort required to collect and monitor data on any further outcomes – both to themselves and to providers and investors.

Hard and soft outcomes

Hard outcomes 

Hard outcomes can be objectively and independently measured. For example, a child being in or out of care or an adult being employed. They can often be underpinned by a statutory process, such as a child being on a Child Protection Plan – there is no doubt about this as it requires a formal decision by a local authority. 

The main advantages are: 

  • there is little room for disagreement
  • easy to measure
  • often the measurement data already exists 

The main disadvantages are: 

  • when used in isolation they can give an incomplete picture of the value of the outcome to the individual 
  • they can bias the measurement towards outcomes that are not significant to the experience of the individual, e.g incentivising a child staying out of care assumes that they will be better off in the long-run staying in the family
  • they may not capture the sustained impact of the service. E.g. a young person may re-engage in education but may drop out after a short period 

binary outcome is a type of hard outcome that has only two states, either an outcome is achieved or it is not. They are used where it is deemed unacceptable for the public sector to pay for outcomes that include negative events. For example, commissioners may run a project to reduce re-offending, and may wish to track its reduction – but this is not binary, and offences will still have been committed. Commissioners should consider the political impact of the measurement and whether a binary outcome may be most appropriate. 

Soft outcomes 

Soft outcomes depend on measurement which is more subjective, e.g. an individual’s self-assessment. They are nearly always measured at the level of the individual or family rather than across the cohort. Soft outcomes are usually set and measured by referring to proprietary measurement methods, such as Outcomes Star family of tools or Strengths and Difficulties Questionnaire (SDQ).

The main advantages are: 

  • they can indicate progress towards a hard outcome that may take time to achieve
  • they can add a richer picture of what is going, compared to using solely a hard outcome which may be misleading on its own

The main disadvantages are: 

  • The quality of the measurement tool used can vary. There are many hundreds of measures out there for soft outcomes, and selecting the most suitable and robust one is difficult and time-consuming.
  • There is increased risk of the measurement being skewed (or even deliberately distorted) by the way the assessment is carried out. 

Below are two examples of setting outcomes and how they divide into hard and soft. Both these examples are from our case studies that provide in depth look into many aspects of the projects in question. 

West London Zone

The West London Zone  supports disadvantaged young people to flourish in all aspects of their life. They have link workers in local children’s centre, schools and employment agencies to help achieve these outcomes: 

Hard outcomes – Increase in school attendance, improved attainment in English and maths

Soft outcomes – improved social and emotional skills, peer relationships and conduct. These are measured through an improvement in SDQ score (strengths and difficulties questionnaire) 

Ways to Wellness 

The Ways to Wellness SIB, commissioned by Newcastle Gateshead CCG, measure improvements in the management of long term conditions achieved through social prescriptions. 

Hard outcomes – reduced hospital admissions, reduced use of outpatient and A&E services 

Soft outcomes – improved wellbeing, measured through Triangle Consulting’s Wellbeing Star

Proxy outcomes

As identified earlier in our contract triangle, it is important that the outcomes you measure (and incentivise) are closely linked to the policy intent of the project. One way to do this is through so-called ‘proxy outcomes’.

Proxy outcomes have been defined by The Centre for Government (US) as an “indirect measure of the desired outcome which is itself strongly correlated to that outcome used when direct measures of the outcome are unobservable and/or unavailable”.

As the name suggests, a proxy outcome acts in place of the actual outcome you are interested in. It is something that tends to go hand-in-hand with the desired outcome – if one is achieved, the other is achieved and vice versa. But beware - correlation does not mean causation so you may end up paying for outcomes which have little to no effect on the actual policy intent of the outcomes-based contract.

Many soft outcomes and measures are used as a form of proxy on the basis that they will indicate achievement of another outcome that is difficult to measure. 

There are many examples of proxy measures and how they can help when outcomes are not easily measured, but there are challenges too. 

  • Children leaving care- the primary outcome is that children leave care and have secured stable and independent living. The proxy outcome can be increased resilience. This is challenging to measure exactly as it is subjective (a ‘soft’ measure), but it can be one of many indicators that the child is ready to leave care.
  • Long term health conditions– the primary outcome is that people are managing their long term health conditions well. The proxy outcome is a reduced hospital admission. The downside to this is that reduced hospital admissions may be due to other conditions having higher priority, or external factors such as longer waiting times. It does not necessarily indicate a positive action
  • People who are unemployed– the primary outcome is that people will find secure employment. The proxy outcome is a reduction in benefits. However, it may be the case that people are no longer be claiming benefits without finding work.
  • People who have been in prison– the primary outcome is that there is a reduction in reoffending. The proxy outcome may be reduced conviction rates. However, many offences may go undetected so it is likely to underestimate both the scale and potential severity of offences committed. 

Lead/progression outcomes

Lead or progression outcomes show movement towards a later outcome.This may be necessary if the primary outcome take a long time to measure and commissioners want reassurance in the meantime that the outcome is likely to be achieved.

For example, for children with highly complex needs in foster care, a lead measure might be better attendance at school, where the evidence indicates this increases the likelihood of the child maintaining a stable foster care outcome. There is also value and impact for the individual and the service from that outcome in its own right. The boxes below offer more examples of how progression outcomes can lead to other greater outcomes. 

Lead/progression outcomes

Lead or progression measures are also used when there is a low likelihood of achieving the primary outcome for the entire cohort, because people have different levels of need. Therefore, some people many require a different starting point and the acknowledgement that are meeting some intermediary outcomes along the way, as indicated by lead or progression measures. 

In an outcome based contract, where payment is placed against the achievement of outcomes, payment against these lead or progression measures may help to mitigate the risk of “parking”, whereby the provider chooses not to work with certain individuals who are too far from achieving the outcome. Doing this can considerably complicate the design of the payment mechanism, though.

The main disadvantage of lead or progression measures is that providers may not invest in moving the cohort towards the primary outcomes if the payment terms provide significant reward at this earlier stage. The level of payment attached to such outcomes, and the number that can be claimed across a cohort, needs to be set with care.

Chapter 4

Measuring outcomes

3 mins read

Commissioners should carefully consider how an outcome will be measured and how the data needed to support each measure will be collected, reviewed and reported. This section will set out a checklist for measuring outcomes and then go into greater detail about the data collection method, and whether you should measure outcomes across a cohort or for an individual.

Checklist for measuring outcomes

Here are many of the questions the commissioner will need to answer when measuring outcomes: 
  • What data will you use to measure outcomes? Is there data available from other sources to measure outcomes? eg. internal performance management, school attendance registers  
  • If data is not available, does there need to be significant investment for new collection processes and systems?
  • who will be responsible for collecting the data and do they have the capacity to do so?
  • If someone else is collecting data, does the data need to be independently checked and validated?
  • Will outcomes will be measured for the individual or across the cohort? 

As measuring the outcomes is about knowing what the impact of the project is, it is important to consider evaluation at this point. Refer to our technical guide to understand what you need to consider for your evaluation. 

Data collection

In terms of measuring outcomes, the method of data collection is crucial. You will need to find out whether the required data is already collected for other purposes. Assumptions should not be made about the availability of data from other parties or the ability of those parties to collect data on your behalf. If certain data is needed to measure an outcome, its availability, the legality and practicality of collection should always be confirmed. 

The table below sets out the types of data and the pros and cons of each approach. 

Data collection

Cohort and individual measurement

The success of the contract in achieving better outcomes (and triggering payment where appropriate) can be measured in two different ways: the individual (or family unit), and the cohort. 

If you are designing an outcome based contract, choosing individual or cohort measures will impact on the payment structure. The majority of outcome-based contracts have used a rate card linked to individual outcomes because this is cheaper to manage, and can be simpler than a cohort level measurement approach. To understand more about this, look at the payment mechanism guide 

The commissioner needs to consider which is most appropriate for the characteristics of the cohort and this is highlighted below.

Individual outcome measurement

Works best with a homogenous cohort. When all members of the cohort experience the same or similar adverse outcomes the outcome target can be set at an individual level. E.g. avoiding entry to care for a cohort who are all at risk of entering care.

A comparison group is not required. As the individual is assessed and this directly relates to the outcomes there is no need to set up a comparison group. Changed or improved outcomes can be specified on a rate card that shows the payment amounts per individual when an outcome is achieved. 

Does not allow for what would have happened anyway (the ‘deadweight’). If there is no comparison group, the outcomes achieved are likely to include some that would have happened anyway, without the intervention (either because of other positive things happening in that individual’s life, or due to natural statistical variation). This can only be avoided if the commissioner has a reliable way to estimate what would have happened anyway (the deadweight), or is certain that the intervention is being applied only to people who would not otherwise achieve an improved outcome. In an outcome based contract, if this is not done, there is significant risk that the provider/investor will be rewarded for achieving too little impact., so the payment level should be adjusted downwards accordingly.

Cohort outcome measurement

Works best if adverse outcomes vary considerably. It is more difficult to set standard measures of success at the individual level if the cohort is made up of different individuals. E.g. the frequency and severity of offending can vary widely and a rate card could not be used.

Usually requires a comparison group – this is needed so the outcomes achieved through the contracted intervention can be compared against a group up did not receive a service. This is known as a counterfactual and is there to check that the outcome would not have happened without the intervention. In an outcome based contract, this can be used to assess whether payments should be made.

Does not require separate calculation of what would have happened anyway (“deadweight”)– any deadweight will be reflected in the measurement against the comparator group, and only performance over and above the comparator group will be rewarded.

Example cohort vs individual

The One Service SIB in Peterborough used a cohort measurement. You can find out more about the methodology used to measure the impact of the SIB project here.

The Innovation Fund SIB by DWP used an individual measurement. This was done through a rate card whereby the price was set for each outcome. You can read more about rate cards in the payment mechanism guide. 

Example cohort vs individual

Chapter 5

Setting the right outcome targets

6 mins read

Commissioners will need to develop outcome metrics so they can measure how success will be judged and when payments will be made. If outcomes have been well written, setting appropriate metrics should be relatively straightforward. The metrics should align to the social objectives of the contract and to the anticipated financial benefits. 

This chapter will set out a checklist for setting outcome metrics, before going into greater detail about the considerations that need to be made. Note that this should not be viewed in isolation as when outcomes are set for an outcome based contract, there will need to be decisions made about payment (see payment mechanism guide). Also, as measuring outcomes are about understanding the impact of the SIB project, you will need to think about how you will evaluate the project (see An Introduction to Evaluation)

Checklist for setting outcome targets

There are several considerations that need to be made when setting outcome metrics:

  • Is there evidence about the level of performance that can be reasonably achieved? 
  • Would stretch targets be appropriate? 
  • Have providers and social investors been involved in the decision-making process? 
  • Do the outcome measures avoid or mitigate perverse incentives? 
  • Have you considered any uncontrolled external factors that have the potential to impact on the outcomes measurement? 

Achievability and stretch targets

Where a commissioner is uncertain whether the performance level they are requiring is reasonable and attainable, they should test their thinking with providers (and investors, if it is a social impact bond contract). There may be a question as to whether there should be stretch targets for providers under a contract. This is only advisable if they are achievable. If they are uncertain whether a stretch target is achievable they can test their thinking with providers and investors during the development and implementation stages.

In a contract that incentivises skills attainment by young people, there may be a metric (and payment) relating to:

  • the young person engaging in skills training (a lead/progression metric)
  • the achievement of a first Level 3 qualification (the primary outcome metric)
  • the achievement of a second, level 4 qualification (a stretch metric).

In a contract that rewards the achievement of employment there may be similar metrics relating to the person:

  • engaging with the programme (a lead/progression metric)
  • being in full-time employment for 13 weeks (the primary outcome metric); and
  • sustaining full-time employment for 26 weeks (a stretch metric, looking at the outcome being sustained)

Involving all stakeholders

As in many aspects of contracting, it is important that the decisions made regarding the outcomes framework are tested with potential service providers and, where appropriate, investors.

This applies especially to outcome based contract where outcomes form all or part of the payment, where it could be argued that ensuring the outcomes framework is acceptable to all parties is the single most important consideration. If the commissioner is prepared to pay for the achievement of an outcome, and providers and investors are comfortable to have success measured by the achievement of the same outcome, a successful contract is likely. Conversely, if one or more parties are not happy with the proposed outcome measure or the performance level required to achieve payment, it will be very difficult to conclude a contract successfully.

Commissioners may wish to consult other parties while designing the service, and should do so prior to any competitive process. This will judge how acceptable the proposed framework is. It will also be useful to benefit from providers’ (and investors') previous experience of similar contracts and what outcome measures and metrics were used then.

Gaming and perverse incentives

When writing outcome metrics, it is important to come up with ways to avoid or mitigate perverse incentives. It is not only outcome based contracts that are prone to these (although they may be especially vulnerable) - to some extent, any effort to measure outcomes is at risk of this. Perverse incentives encourage contract stakeholders to behave in a way that is detrimental to contractual goals even if some outcome metrics improve. The result of perverse incentives can be negative consequences for service users. They can occur for all parties of a contract, not just providers or investors.

There are two main forms of perverse incentives, these will be discussed below with examples and ways to mitigate them.

1.    Exerting influence on who is eligible for the service - ‘cherry picking’

Sometimes providers, investors or intermediaries may select beneficiaries that are more likely to achieve the expected outcomes and leave outside the cohort the most challenging cases. This is known as ‘cherry picking’.

An example of this is when an outcome is chosen that seeks to achieve an absence or reduction of referrals to a statutory agency. The perverse incentive is to simply not refer people even if they fit the criteria for statutory support. This will make it look like there has been a reduction even though it is not in the best interests of the beneficiaries. 

To mitigate this there should be a collective decision-making process involving both the commissioner and service provider, or a neutral referral party/mechanism. This provides the opportunity for all parties to make and agree decisions that are in the best interests of the beneficiary and avoid ‘cherry picking’.

2.    Choosing who to work with in the cohort - ‘creaming’ and ‘parking’

In some instances, providers may choose particular people from the cohort who are already nearer to the desired outcome and therefore are easier to support – this is known as ‘creaming’. The opposite of this can also happen whereby providers neglect people who are less likely to achieve positive outcomes – this is known as ‘parking’. 

This may occur when an outcome target does not consider the varying levels across the cohort. For example, an outcome measure may be ‘an employment support programme requires 50 number of people to be in employment in 12 months’. The cohort may be made up of people who have very different needs. More challenging cases such as those with complex needs and who are longer term unemployed may be ‘parked’ and those who are likely to find it easier to get a job may be supported more. To mitigate this the outcome metric could be structured so that the start point of the individual is recognised and they are rewarded for the progress made. (See lead/progression outcomes.) This will mean that variation in the cohort is accounted for. 

It may also occur when there is a simplistic binary outcome that results in no support being given after a certain milestone – a ‘cliff edge’. For example, a homelessness project may seek to settle 20 many homeless people in 12 months. If people fail to achieve this outcome they are no longer of value to the provider and are ‘parked’. To mitigate this, the target should reward success at small and regular intervals, or offer bonus payments. The amount of homeless people settled could be measured at 12 months, 18 months and 24 months and bonus payments could be set to encourage people to settle in accommodation.

You will need to consider how you will mitigate against perverse incentives so that the outcomes are in the best interests of the beneficiaries. If you refer to the contract triangle and make sure you do the three key points well, you should avoid perverse incentives.  

Mitigating the impact of uncontrolled external factors

When setting outcome measures it is advisable for commissioners to avoid measures that can be unduly influenced by factors outside their control. It may be impossible to identify social outcomes that are entirely impervious to wider changes in social policy, or changes in demographics or labour market change. However, it is wise to avoid outcomes that can easily be under or over achieved due to external factors. These are: 

  • possible changes in government policy, eg. an outcome relating to educational attainment might be affected by changes in the national curriculum 
  • planned service changes, eg. an outcome from an intervention focused on children in need may be affected by the reorganisation of a Children’s Services Department. 
  • Changes in local practice, eg. plans by another body to commission a similar intervention at the same cohort in the same area may undermine the validity of a comparison group 

Whilst these external factors may not be under control, it is important to determine the potential impact they could have when designing the service.  It is wise to take into account the likely influencing factors in the metrics, they could be both positive or negative. It is also wise to have agreements with other practitioners so they all operate in a way that enables the outcomes to be achieved. 

In some circumstances, commissioners may accept obligations under the contract to manage the impact of these factors, but this is often not the case. It may be preferable to include the monitoring and reporting of these impacts as part of the contract management process, with the expectation that the commissioner will take steps to mitigate or avoid where possible.

Examples of setting outcomes and targets

If you look at our case studies you will find examples of outcomes measures and targets that have been designed in a robust way. 

Chapter 6

What does 'good' look like?

2 mins read

As there is no ‘correct’ or ‘set’ way to create outcomes, measures and targets you may be questioning what good looks like. Having understood the considerations that have been outlined in this guide you are equipped to write a robust outcomes framework. 

You can test your outcomes using the simple decision tree below. Once you have identified the outcomes and their appropriate measures you can check the outcomes definition section of the SIB readiness framework to see what best practice will look like. 

Figure 2: Suitable outcomes
Figure 2: flow chart to identify suitable outcomes

West London Zone designed a robust outcomes framework and this section will paraphrase from their website. 

As part of the Collective Impact Bond Steering Group process, WLZ worked with key stakeholders in the area to analyse some of the challenges that children and young people growing up here face. The goal was to understand where to focus efforts, and what outcomes we should be aiming to achieve over the long-term. Many areas were identified at each age stage.

WLZ took these and distilled them down into an ‘outcomes framework’, which intends to show the progress a young person is making towards flourishing in all areas of their life. WLZ has attempted to find a way to reliably track the ‘whole child’.  The WLZ whole child outcomes framework is enables all those working with a young person to understand and track their progresstowards flourishing on an ongoing basis.

The framework covers a total of eight ‘outcome areas’ across the three domains of wellbeing, learning and character. All eight are measured and monitored for school-age children, but a lower number are measured and monitored in very young children and young adults as they are not all relevant. Each of the ‘outcome areas’ has one or two measures which enable the monitoring to take place 

Data on each of these metrics will be collected annually through the WLZ Data Collection Survey as well as through secondary data already collected by schools and, where appropriate, other statutory sources. Some will be collected more regularly through delivery organisations. 

West London Zone outcomes

You can view more examples of outcomes frameworks on the case studies as part of our website. 

We have designated this and other guides as ‘beta’ documents and we warmly welcome any comments or corrections. We will regularly review and update our material in response. Please email your feedback to