chevron icon twitter logo facebook logo linkedin logo Youtube logo download icon quote icon posted icon clock icon author icon arrow icon arrow icon plus icon Search icon location icon location icon document icon plus-alt

Our research work

Situation and challenge

The contracting and commissioning out of public services has become increasingly focused on outcomes – the positive results that services produce in the lives of services users and citizens. This marks a shift in the traditional way of doing things, where government pays for public services on the quantity of activities (e.g. number of job skills courses offered) and outputs (e.g. attendees in a job skills course) (Bovaird and Davies, 2015).

The compelling logic within outcome based commissioning (OBC) – that specifying and steering services on the basis of social outcomes will deliver better outcomes – is deliciously straightforward and aligns comfortably with much of the literature undergirding the advancement of performance measurement and management in public service contracting and delivery. Officials celebrate these outcomes-oriented models for being highly effective and uniquely innovative, but attempts to reengineer public service decision-making around outcomes have been around at least since the 1980s (Pollitt and Bouckaert, 2004). Since 2010, OBC has been promoted with increasing vigour by the UK Government as a way to improve cost effectiveness, inspire innovation, boost accountability, incentivise systems-level planning, and inspire responsiveness, with risk transferred to the private sector (NAO, 2015; Brown, 2013). In recent years, the practice has taken on increasing international significance at the leading edge of public service reform (Farr, 2016, Garton Grimwood et al., 2013; Cabinet Office, 2011).

At the same time, evidence supporting the effectiveness of schemes operating under OBC is alarmingly limited (Fraser et al., 2016). While there are many ways to ‘do’ outcomes-based commissioning (OBC) (i.e. payment-by-results and social impact bonds), there has only been modest success in shifting commissioning focus from inputs and outputs to outcomes (Bovaird and Davies, 2015). 

As a newly established centre for research and commissioner support, this is the challenge to which the GO Lab responds. We seek to make significant and substantial contributions to academic literature, and crucially, to the wider commissioning landscape by enabling practitioners to make informed decisions about which commissioning approach is best, using the best available evidence.

This section sets out the high-level agenda for the research team. 

1. Understanding what is meant by OBC and the expectations associated with this approach

1.1 Rationale

Existing research sets out a range of motivations for pursuing OBC (Fraser et al., 2016; Cabinet Office 2011; Ronicle et al., 2014). Indeed, the multiple justifications for adopting the approach are often positioned as part of OBC’s ability to align the interests of multiple, diverse constituents. What is less clearly articulated, however, is the extent to which these alternative justifications or ‘ends’ may compete, be irreconcilable with one another, or be prioritised to different degrees in alternate applications of an outcome-based approach.

The reason it is important to nail down what OBC is for is that, without coherent exposition of why OBC is being pursued, there will be considerable ambiguity in the development of planned research, particularly when framing questions of impact evaluation. In the absence of an understanding of what varying examples of OBC are seeking to deliver, there will be great challenges in nailing down an assessment of whether the approach is succeeding ‘on its own terms’.

The conceptual unpacking of OBC within this strand then ought not to be a dowdy or simplistic process of definition setting. Rather, the challenge is for active engagement with academics and practitioners in order to get inside the concept and understand its varying intents and logics. 

Moreover, tracing the intellectual provenance of OBC will facilitate a more nuanced consideration of whether OBC enables the reconciliation of perennial issues in public administration, particularly in relation to the steering of principal-agent relationships and the squaring of efficiency, effectiveness, and equity concerns.

1.2 Overarching research questions

  • What are the logics and assumptions underpinning the application of OBC?
  • What is the problem to which OBC is the solution – i.e. if OBC is the ‘means’ then what are the ‘ends’ to which this approach is being leant?
  • What is OBC’s intellectual provenance and does it mark a step change in relation to previous practices?

2. Assessing the impact of OBC in relation to the ‘expectations’ associated with the approach

2.1 Rationale

A dominant justification for OBC is undoubtedly its perceived ability to deliver more and better outcomes for programme participants than conventional commissioning and management approaches. There is, however, a paucity of evidence in relation to the tangible improvement in outcomes associated with OBC implementation, and very few robust impact evaluations exist (Lagarde et al., 2013). To date, there is no compelling evidence in support of the superiority of outcomes-based approaches, and existing studies suggest that the use of private or voluntary sector providers is no guarantee of superior performance (Fraser et al., 2016). Where previous outcome-based programmes have been evaluated, they have shown mixed success (NAO, 2015).

Arguably, OBC is at a critical juncture: evidence of delivering on its promises is necessary as attitudes amongst Whitehall civil servants begin to ‘cool’ (Brown, 2013). In order to avoid death-by-scepticism, OBC needs robust, independent impact evaluations across a range of contexts and scales. 

It is only once the approach is associated with demonstrable positive impacts that we can viably pursue OBC as evidence-based practice.

2.2 Overarching research questions

  • Has OBC made a tangible difference to the delivery of outcomes? What are the impacts of services commissioned through OBC, compared to more conventional commissioning approaches?
  • Do ‘outcomes’, as specified by commissioning agreements, have meaning in the lives of programme participants and citizens?
  • Do identifiable aspects of context, organisation, or intervention mediate OBC impacts?

3. Understanding the mechanisms and relational dynamics of OBC

3.1 Rationale

Research, at present, offers a relatively crude set of ‘necessary conditions’ for OBC to be a viable commissioning strategy. The largest evaluation of SIBs to date (Ronicle et al., 2014) sets out several ‘benefits’ of the approach, four of which could be considered potential routes through which the mechanism offers improvement on previous models, for example:

  • Focus on outcomes – where refinement of objectives unlocks provision that is more ‘innovative’ and responsive to need;
  • Better contract management – derived from clearer statement of policy objectives, improved management information systems and the squeeze of tighter contract management;
  • Alignment of interests – where all parties are ‘pulling in the same direction’ and working collaboratively when it comes to outcomes delivery;
  • External investment – with performance benefits from the support and performance management expertise associated with external, private capital.

These potential (and any additional further) dimensions have yet to be explored in any depth. Each dimension would benefit from further unpacking prior to the design of qualitative longitudinal evaluation. For example, while the ‘alignment of objectives’ is typically described as alignment across commissioner and service provider (and investor where present), there are potentially four tiers through which an outcome-based approach may offer a fundamental shift in relational dynamic away from procedural and contractual levers to a dynamic of cooperation and co-production:

  • Alignment across multiple commissioner organisations through budget pooling and processes of ‘co-commissioning’;
  • Shared objectives and collaboration, as conventionally described in social impact bonds between commissioners, providers and investors;
  • Collaboration and alignment across multiple service providers within a holistic, networked ecosystem of support (as in Peterborough’s ‘One service’ or ‘Working Well’ in Greater Manchester);
  • Co-production and shared value creation between services user citizens and frontline staff.

3.2 Overarching research questions

  • What are the technical, institutional, and cultural enablers and barriers to successful OBC?
  • What are the obstacles to the wider adoption of OBC approaches?
  • How have practitioners overcome working challenges associated with ‘doing’ co-commissioning and collaboration? What types of governance arrangements and capacities facilitate success when outcomes are shared and leadership is distributed across a system?

Questions on how best to ‘do’ OBC should be predicated on research findings from strand 2 and consideration of whether OBC stands up as a desirable model.

For more information about our work and how to get involved, you can get in touch with us at golab@bsg.ox.ac.uk