chevron icon Twitter logo Facebook logo LinkedIn logo YouTube logo download icon link icon audio icon quote icon posted icon clock icon author icon arrow icon arrow icon plus icon Search icon location icon location icon document icon menu icon plus-alt

In 2019, this tool was developed in partnership with Ecorys, in consultation with a diverse range of social outcomes contracting practitioners, for the UK Government’s Life Chances Fund. It may be applied to other impact bonds but users should consider differences in context.

1. Review options

22 minute read

1.1 Define the problem

Step 1 of 5

Have you investigated the problem and the wider system it fits in?

Things are going well when …

You have focused on a social problem and have investigated and identified the problems with current services, or the gaps in services.

You have explored reasons for the problems or gaps, and have distinguished between the social problem itself and the issues within the system or ecology of services that might have prevented it being tackled to date. For example, fragmented services, reactive spending and/or difficulty in changing underperforming or outdated ways of working innovating.

You may have cause for concern if …

There is ambiguity about the social problem you wish to tackle, or there is no clear distinction between the social problem itself as opposed to the systemic issues which might have caused it or prevented it from being tackled. The problem you are trying to solve is not best addressed by a particular service stream and is difficult to measure robustly.

1.2 Gather learning

Step 2 of 5

Have you gathered evidence around the problem and how to tackle it?

Things are going well when …

You have gathered learning to inform your analysis of the problem and options for tackling it, by reviewing your own learning and evaluation material, reviewing evidence from elsewhere, speaking to others who may have relevant experience, and gathering the opinion of current users of services.

You may have cause for concern if …

You have not gathered adequate learning to fully analyse the problem or options for tackling it. Learning gathered/ available data does not indicate front-runner solutions or provides conflicting information.

1.3 Appraise different options

Step 3 of 5

Have you considered the costs and benefits of different solutions?

Things are going well when …

You have developed options for using different solutions to address the defined problem.

You have appraised the costs and benefits of different options, both in monetary and non-monetary terms.

This has enabled you to gain a clear understanding of the potential of Impact Bonds alongside other alternatives – for example, using outcome-based payment without social investment, or having a greater outcomes focus in current grants or service contracts.

You may have cause for concern if …

Alternative options have not been properly considered, or their costs and benefits have not been fully understood.

1.4 Choose the best alternative

Step 4 of 5

Can you explain why an impact bond the best option?

Things are going well when …

Determine whether to use an impact bond through analysing if the benefits might outweigh the costs compared to other approaches. Keep in mind that benefits outweighing costs can apply to both monetary and non-monetary terms.

Consider whether anticipated benefits of impact bonds might apply: helping to promote collaboration between outcome payers and service providers, increasing the focus on prevention in spending, or triggering innovation in the way services are delivered.

You may have cause for concern if …

There is no clear justification for the use of an Impact Bond over other alternatives.

1.5 Consider viability

Step 5 of 5

Have you assessed viability by using the LOUD model?

Things are going well when …

You have determined whether an impact bond model might be viable by assessing your circumstances against the factors identified in the LOUD model: Leadership, definable Outcomes, shared Understanding, Data. In addition, the provider ecosystem and favourable regulations are key factors in enabling successful development of impact bonds.

You have a clear understanding of the resources required to develop the Impact Bond model, and where external support may be required, and you have budgeted for this.

You may have cause for concern if …

It is not clear at the outset whether an impact bond model is viable in the circumstances.

2. Develop the business case

2.1 Target cohort

Step 1 of 5

Can you evidence why have you chosen your target cohort?

Things are going well when …

You have clearly communicated why you have chosen the target cohort and defined its characteristics (e.g. socioeconomic, demographic, incidence of the social issue). This is supported by an evidence base and stakeholder engagement.

You may have cause for concern if …

You have poorly defined your cohort. There is no clear process of identification and there is ambiguity over the participants.

A common challege that outcome payers have experienced

Facing ideological opposition or suspicion about SIBs from key decision-makers due to wariness of perceived 'privatisation' of public services.

Lack of understanding about SIBs, including limited skills and knowledge, and a lack of time and resource to gain the skills and knowledge needed. This makes it difficult for commissioners to communicate the rationale for developing a SIB to other stakeholders.

Concerns about the potential risks involved.

How some outcome payers have overcome this

Some project leads who faced scepticism emphasised the longer-term benefits. For example, in one SIB, the project lead stressed that the SIB could be used to test an initiative, which, if successful, could be rolled out later as part of the core business.

Project leads found it helpful to build in sufficient time and resource to the process to ensure that they are knowledgeable about – and confident to – communicate the feasibility and appropriateness of the SIB to key decision makers. They found that many of the more technical aspects could be learnt as they went along, either by using tools or guidance, or by drawing on advisors’ expertise.

Several project leads highlighted that it was important to carefully consider the risks to their organisation in commissioning a SIB. If the potential reputational risk could be outweighed by the positive outcomes, then it was important that this was communicated to key decision makers.

2.2 Outcomes definition

Step 2 of 5

Have you defined outcomes that have verifiable measures?

Things are going well when …

You have chosen a small number of clearly defined and specific outcomes that align with policy intent.

Each outcome has appropriate and measurable indicators that can be independently verified.

You may have cause for concern if …

The outcomes have been poorly defined and they are not capable of being measured or verified

A common challege that outcome payers have experienced

Striking the balance between easily measurable outcomes, and outcomes that are sufficiently personalised to the beneficiaries

It can be time consuming to agree on outcomes

How some outcome payers have overcome this

Some projects leads overcame this issue by building in sufficient time at the beginning to allow for an iterative process of defining outcomes.

Project leads emphasised the importance of keeping outcomes simple (opting for a few key outcomes rather than many outcomes), but not at the expense of excluding important outcomes.

2.3 Price of outcomes

Step 3 of 5

Have you set a price for outcomes and can justify it?

Things are going well when …

There is a clear articulation of price of outcomes and when they will be paid (payment milestones).

There is a strong justification based on an existing evidence base, which may be financial or historical data.

These outcome payments have been sense checked by the other stakeholders, namely providers/investors.

You may have cause for concern if …

There is an unclear rationale for pricing of outcomes, or it is unclear how and when payments will be paid.

A common challenge that outcome payers have

Having a lack of confidence and knowledge on pricing outcomes, due to not having the need to do it in the past.

How some outcome payers have overcome this

Many successful project leads spent time at the beginning of the process to review how previous SIBs priced outcomes, exploring existing sources of evidence (see ‘Links to Resources' box), or by speaking to other commissioners to get the data.

Many project leads engaged an advisor who knew how to navigate existing data on pricing outcomes, or how other SIBs priced outcomes, based on their previous advisory work.

2.4 Financial model

Step 4 of 5

Does your financial model work in different scenarios?

Things are going well when …

The financial model has been developed and this shows the financial effect of different scenarios for both the volume of participants and their success at achieving outcomes and payment milestones.

All stakeholders (outcome payer, investor, provider) understand and support the financial model and understand the implications in terms of budgeting / financial planning / cash flow.

You may have cause for concern if …

The financial model does not show the effect of different scenarios regarding volume of participants and/or success at achieving payment milestones.

The stakeholders either do not understand or do not support the financial model.

A common challenge that outcome payers have

Knowing how to actually undertake financial modelling and having the skills in-house to do it.

Agreeing on a financial model that suits the needs of commissioners and investors.

How some outcome payers have overcome this

One project lead undertook the financial modelling early on, as a way to calculate which outcomes would be viable. This helped to save time, because they started with draft outcomes definitions and values, and used the modelling to refine them (rather than defining and valuing outcomes first, and then finding out they were not financially viable).

Project leads emphasised the importance of making sure that they factored in a range of potential costs, including wider costs that might be borne by the local authority (such as performance management).

It was important for some project leads to ensure that there was a consensus among stakeholders on how rates of return were expressed.

2.5 Investment requirement

Step 5 of 5

Does your financial model show minimum and manimum investment required?

Things are going well when …

There is sufficient information to enable investors to derive a minimum and maximum investment requirement and when investment will need to be "drawn down".

Investors can be made clearly aware of the responsiblities and risks of backing the project.

You may have cause for concern if …

Investment amount and timetable is not known, nor the risks and responsibilities created by the project.

3. Managing relationships

3.1 Lead outcome payer role

Step 1 of 4

Are you aware of risks, interdependencies and the needs of stakeholders? Have you planned for these?

Things are going well when …

A senior decision-maker has signed off the development process and an accountable person has been appointed.

A project planning group is in place to oversee development of the project.

You understand and have mitigated risks before and during the development process.

You are aware of interdependencies and stakeholder management needs of the project and will activate any actions if needed.

You have made a plan for the development process that included interdependencies and stakeholder management.

You may have cause for concern if …

There is a lack of clear leadership of the project, or evidence of commitment from the required internal stakeholders within the outcome payer's organisation.

Roles and responsibilities are not clearly assigned.

A common challenge that outcome payers have

Securing internal buy-in when there is misunderstanding or scepticism from colleagues about what a SIB is.

Churn within the commissioning organisation, making it difficult to maintain working relationships with different teams, such as finance or legal.

How some outcome payers have overcome this

Some project leads engaged key decision makers early on, and spent time to explain what the SIB was and why it was beneficial to their organisation (over and above other approaches to commissioning)

In some SIBs, project leads developed a working group comprised of staff from different teams within their organisation (including finance, legal, procurement, as well as service leads) from the outset. In one area, the working group met on a regular basis to ensure that all teams were kept abreast of developments, so any emerging issues could be dealt with promptly.

Some stakeholders highlighted it was helpful to keep written documentation of the progress and decisions made throughout the SIB process because it meant that any new stakeholders could get acquainted with the SIB quickly.

3.2 Provider market collaboration/dialogue

Step 2 of 4

Have you engaged the market to identify providers?

Things are going well when …

As an outcome payer, you have engaged with the market to identify new and existing providers. If needed, market development and testing has taken place, including capacity building of service providers and positive feedback on proposed terms.

You may have cause for concern if …

There is no capability or capacity of the provider market (i.e. service providers, often charities or social enterprises), or this has not been tested or understood at large by the service providers.

You feel that you are not able to choose the best provider(s) as you have limited options.

A common challenge that outcome payers have

Explaining the SIB mechanism to providers, who might not be familiar with it

Explaining the rationale for running the intervention through the SIB mechanism, as opposed to a more familiar mechanism, such as fee-for-service or grant

How some outcome payers have overcome this

Some project leads engaged providers early on (after developing the technical aspects of the business case) to give them time to get a better understanding of what the SIB mechanism was, and how it might have affected them. It was also important for project leads to explain to the providers the reasons for running the intervention through a SIB.

One contracting authority engaged providers through holding ‘market testing’ days, where they invited prospective providers to learn about the SIB and discuss how best to set the procurement specifications. This worked well because it ensured all providers were on the same page, and it resulted in many interesting and high quality proposals.

3.3 Investor engagement

Step 3 of 4

Have the investors been engaged from the start?

Things are going well when …

Investors have been engaged in the development of the model since the start. The investor market and its representatives (e.g. intermediaries) have worked through the financial model and fully understand it.

You may have cause for concern if …

Investors are only engaged in the later stages of the project.

There has been no testing of the expected target level of outcomes or price which influences the financial and impact proposition for investors.

There has been no testing of the investment required with investors or its representatives (e.g. intermediaries).

3.4 Procurement strategy

Step 4 of 4

Which procurement approach are you taking?

Things are going well when …

The procurement approach has been decided (e.g. restricted process, open competitive process with dialogue, etc). The procurement considerations are appropriate, proportionate for both the funding mechanism and the service, and consider the implications on governance and service delivery.

You may have cause for concern if …

There has been no consideration of suitability of the procurement approach, or of the implications of this on the wider stakeholder engagement required and the project as a whole.

A common challenge that outcome payers have

Overcoming potential conflicts-of-interest in the design and commissioning process, particularly in provider-led SIBs.

Operating in often rigid structures of commissioning organisations’ standard procurement practices.

How some outcome payers have overcome this

Some project leads involved the procurement team in the SIB development at an early stage, as part of a working group. This allowed procurement teams to advise about the process of provider market dialogue and collaboration.

For some project leads, it was important to review existing guidance and documentation (see ‘Links to Resources’ box) at the start of the process, so they could be aware of the different routes to procurement that could be taken.

4. Design the service

4.1 Referral or identification process

Step 1 of 5

Does your referral pathway avoid perverse incentives? Have you considered regulatory or statutory responsibilities?

Things are going well when …

A referral pathway has been developed (e.g. voluntary opt-in, proactive engagement or referral from another agency entity) which includes appropriate governance to avoid perverse incentives. Considerations have been given to the decision of the most adequate referral pathway.

The referral pathway has been developed in full knowledge and oversight of any regulatory or statutory responsibilities and with stakeholder engagement. A process is in place to determine eligibility based on the defined target cohort.

You may have cause for concern if …

No clear process for accessing proposed project participants.

A common challenge that outcome payers have

Developing referral pathways that prevent the cherry picking of people from whom outcomes might be easier to achieve.

How some outcome payers have overcome this

In one area, the project lead established one route for all referrals to come through, with a specific screening tool built in to identify people with more complex needs. This allowed the contracting authority to have more oversight of the referrals that came in, to ensure that those with more complex needs were being supported through the SIB.

4.2 Intervention model

Step 2 of 5

Are paitipcants involved and consulted in the deisgn and delivery of the intervention?

Things are going well when …

Either…

There is clarity on how to specify and access the desired interventions.

Or…

There is a strategy for co-designng the intervention with stakeholders.

Or…

In the case of a 'black box' intervention approach, a specific defined approach to ongoing contract and performance management is in place.

In all cases, participants are involved and consulted in the design and delivery of the intervention.

You may have cause for concern if …

There has been no consideration of approach to intervention model and the effect of this on the project overall. It is unclear how the intervention model (or process for developing an intervention model) links to the proposed outcomes.

A common challenge that outcome payers have

Choosing whether to have an evidence-based intervention (and obtaining appropriate licences) or having providers design the intervention.

Giving up control over the design of the intervention to providers.

How some outcome payers have overcome this

One project lead found that holding a ‘market testing day’ with providers helped when it came to refining the specifications for the bid. They found that providers were more likely to propose a service that was in line with the contracting authority’s vision, as everyone had the same understanding of the rationale for the SIB.

4.3 Client journey and service integration

Step 3 of 5

Have you articulated how participatns will experience the service?

Things are going well when …

There is excellent articulation of how a participant will experience the service, including the role of the intervention(s).

The participant journey clearly integrates with existing services.

You may have cause for concern if …

Unclear on what the participant journey looks like, or the integration with existing services have not been thought through.

4.4 Evaluation (impact and process)

Step 4 of 5

Do you have a theory of change? What is your baseline and counterfactual?

Things are going well when …

A 'theory of change' to show how the project is expected to achieve improved outcomes has been articulated. There is a demonstrable understanding of the baseline and counterfactual and a clear plan in place for independent data verification and for evaluation of the project.

You may have cause for concern if …

No consideration given to evaluation approach.

Step 5 of 5

Are there any regulations you need to follow?

Things are going well when …

Any regulatory requirements relating to the services, such as Ofsted registration or safeguarding protocols, have been considered and planned for.

You may have cause for concern if …

There has been little or no consideration of regulatory and legal requirements in the context of the proposed funding mechnism and service design.

5. Mobilise for delivery

5.1 Contract signed

Step 1 of 6

Has the contract been signed?

Things are going well when …

An appropriate contract has been signed which satisfies all the areas of development: the business case, relationships and service design.

You may have cause for concern if …

The contract does not satisfy all the areas of development and is therefore not viable.

A common challenge that outcome payers have

Drafting an appropriate contract that aligns key stakeholders’ (commissioners, investors, and providers) interests so all parties are satisfied.

How some outcome payers have overcome this

Some project leads engaged legal teams early on in the procurement process to advise and support on drafting the contract.

One project lead shared the draft contract with prospective providers during the procurement stage to help to speed up the contracting process.

5.2 Stakeholder awareness and buy-in

Step 2 of 6

Do staff know about the service and have they been trained?

Things are going well when …

There has been staff training and clear communication within the outcome payer organisation about the new service and how to utilise it.

External stakeholders, e.g., local service providers are aware of the project and how it integrates with other existing services.

You may have cause for concern if …

Staff have not been trained about the project and there is poor communication within the organisation about how to use the service.

External stakeholders are unaware of how the project integrates with existing services and any promotional oppportunities have not been considered.

A common challenge that outcome payers have

Gaining buy-in from other members of staff within the commissioning organisation like service managers and frontline staff.

How some outcome payers have overcome this

Some project leads engaged with staff through workshops, or set up implementation groups (where frontline staff could advise on how a certain aspect of the SIB intervention could work). For example, in one SIB, the project lead conducted several workshops with staff to allow them to raise any questions or concerns they had about the SIB.

5.3 Processes and resources

Step 3 of 6

Do you know how you will reach the number of referrals needed and have enough resources to do so?

Things are going well when …

Referral pathways or methods of identifying and engaging participants are in place so that the agreed number of referrals will be reached.

There is adequate staffing, resources and skills in place across all parties to launch the service (including logistics like desk space and IT access).

You may have cause for concern if …

Agreed referral pathways and/or engagement methods are not in place to ensure the number of participants will be reached. The required staffing resources or skills are not in place to launch and run the project.

5.4 Governance and management structure

Step 4 of 6

How are you governing and managing the intervention?

Things are going well when …

The roles and responsibilities of the outcome payer, investor and service provider during the delivery are decided (plus intermediaries if relevant).

Appropriate governance and management structures are in place, and meetings scheduled.

Your governance structures allow for insight from ongoing service delivery to be analysed and understood and for ongoing service adaptation.

You may have cause for concern if …

There is a lack of clarity about the roles and responsibilities of the outcome payer, investor and service provider.

The structures to enable project management and problem solving during delivery are not in place.

5.5 Data, reporting and performance management

Step 5 of 6

Have you got a system to report progress? Are you compliant with data sharing regulations?

Things are going well when …

Systems are in place to report on the project progress and achievement of outcomes, including methods to verify and validate data and to tackle data gaps or unavailable evidence.

Payment processes are in place (particularly where there are multiple outcome payers involved).

The project is compliant with data sharing regulations and required data sharing agreements and data consent processes are in place.

Major risks during delivery are identified, quantified and mitigated against.

You may have cause for concern if …

Systems are not in place to measure progress towards outcomes, to report on the project, or to validate data.

The project is not compliant with elevant data protectin regulation and/or data sharing agreements are not adequate.

Risks have not been considered, or have been considered but not adequately mitigated against.

A common challenge that outcome payers have

Having strong performance management processes in place.

Designing or utilising data collection tools that are fit-for-purpose.

How some outcome payers have overcome this

Some project leads had discussions with the investors to identify the best approach to developing performance management systems. This was helpful because investors usually had experience doing this so they were well positioned to advise.

One project lead scoped out the service provider's ability to implement performance management systems. When they found it would not be sufficient, they agreed with the provider that they outsourced the performance management to ensure that it could be completed at the level of quality needed to meet the investor’s requirements.

5.6 Ongoing learning and review

Step 6 of 6

How are you capturing ongoing learning? Does your evaluation integrate learning and share benefits?

Things are going well when …

There are processes and structures in place for input from stakeholders and ongoing learning.

An evaluation is set up which includes a process for integrating learning when the service ends, including realising savings or system-wide benefits where appropriate.

You may have cause for concern if …

Ongoing learning is not facilitated or encouraged, including processes for iterating delivery approaches.

There is no plan to test the benefits of the service delivery, and/or no evaluation is to be conducted.

6. Oversee delivery

6.1 Orientate for delivery

Step 1 of 5

How are you managing the transition from launch to delivery? Who is accountable for contract performance?

Things are going well when …

There is a smooth transition from the team responsible for launching the project to those responsible for overseeing it (contract management)

There is a clear line of accountability for the performance of the contract within the outcome payer organisation

You may have cause for concern if …

Team responsible for overseeing project are inadqueately briefed or do not understand specific requirements of outcome based contract

Accountablity for performance of the contract within the outcome payer organisation is unclear

6.2 Use of data to inform delivery

Step 2 of 5

Is data being used to inform decision making?

Things are going well when …

The provider is encouraged to make use of all available operational and evaluation data to inform decision-making during live running

The agreed reporting structure and reporting cycle is effective, and there is a clear feedback loop between reporting and decision-making, to ensure data is being used for learning and adaptation

You may have cause for concern if …

Data is not being used to inform decision making during live running

The reporting cycle does not assist with decision-making or creates additional work with limited benefit

6.3 Monitor financial progress

Step 3 of 5

Are you tracking performance against the scenarios in the financial model?

Things are going well when …

Actual performance is tracked against the scenarios in the financial model, such that the financial effect of performance on providers and investors is continually understood

Outcomes data for payment claims is accurate and verified as agreed, and discussions take place to problem-solve if data availability is affecting the ability to claim payment

You may have cause for concern if …

Actual performance relative to what was predicted at the start is unknown

Outcomes data is inaccurate, unverified or incomplete, and no action is taken to rectify this

6.4 Collective problem-solving

Step 4 of 5

Do performance management processes allow for changes to be made? Is there clear communication about issues?

Things are going well when …

The outcome payer participates in project oversight, and conducts performance management processes, according to the responsibilities agreed during set-up, and adjusts the approach as necessary according to the level of performance

There is a clear mechanism for ongoing communication between the outcome payer organisation and parties involved in delivery (provider and investor), including to remedy any performance issues

There is an open and collaborative approach to problem solving, together with the provider and (if appropriate) investor

You may have cause for concern if …

Project oversight and performance management is lesser or greater than agreed at the start, or than what is necessary considering the level of performance of the contract

Communication between outcome payer, provider and (if appropriate) investor is difficult

Problem solving is not conducted collaboratively

6.5 On-the-ground implications

Step 5 of 5

Are there adequate resources? Will there be a review of the referral process?

Things are going well when …

The provider can continue to show adequate skills and resources for contract performance, and additional training for staff is agreed, including how to resource it, if required

The referral process is reviewed for effectiveness, and the client journey is well integrated with other services

There is ongoing communication to staff at all levels in the outcome payer organisation about the contract's aims and requirements from them

You may have cause for concern if …

There is a skills deficit within the provider organisation with no mechanism for addressing it

Referral processes are not working as intended, and / or the service does not integrate with existing provision

Staff at any level of the organisation are unaware of the existence, or purpose, of the service

7. Ongoing adaptation and learning

7.1 Post-launch adaption of contract

Step 1 of 2

Are there flaws in the contract that need to be amended?

Things are going well when …

A clear and ongoing distinction is made between poor performance and flaws in contract design.

Flaws in contract design are addressed through negotiation, using an independent mediator if necessary. These might include:

  • The payment mechanism is not driving the expected behaviours
  • Assuptions made around the cohort's 'baseline' or starting point have turned out to be drastically different from reality on the ground
  • The data to evidence the achievement of outcomes is not available as expected
  • External circumstances change beyond the control of any of the parties, such as a new national policy
  • The service is incurring unanticipated costs and there is a case for the outcome payer to assist in mitigating this

You may have cause for concern if …

Poor performance is mistaken for contract design flaws, or contract design flaws for poor performance.

Design flaws are not resolved even once identified and acknowledged

7.2 Long-term considerations

Step 2 of 2

What added value has the impact bond had? What are the implications for the future?

Things are going well when …

The Impact Bond is achieving what was proposed in the business case, or there is good understanding of what additional value the contracting mechanism is adding (for example, is it helping to respond to fragmented services, reactive spending and/or difficulty innovating?), or a clear view is emerging that it is adding nothing

There is an ongoing understanding of the effect on a wider set of outcomes than those being paid for; interim evaluation findings are requested and considered

There is a growing understanding of the effect on the wider system and the wider application of learning, and the implications of this for future contracting decisions

Discussions begin on what follows the end of the contract, including the impact of this on users of the services: renew, repurpose, or re-think the approach

You may have cause for concern if …

It is unclear whether or not the impact bond is providing additional value beyond an alternative method of contracting

Wider benefits or costs, and wider system effects, are not understood or acknowledged

No clear view is emerging on what to do after the contract completes: whether to renew, repurpose or re-think the approach.