Measurement and Economic Validity
Set my timezone
Select a timezone from the map or dropdown and click the Set timezone button to adjust the event date/times to your current timezone.
For the second session, we explored measurement and economic validity. Our guest speakers introduced different approaches in impact measurement and invited the audience to share their experience.
The Value in public finance peer learning group provides a platform for those in government, academia, private and other sectors to discuss and explore ways to improve value creation of public expenditure. This peer learning group aims to create a community of individuals and organisations interested in improving public finance and how economies perform. We are an inclusive group of thinkers and practitioners and would welcome international engagement on the above themes as well as any others.
The group meets every 2 months for an hour-long discussion and is co-organised by the GO Lab and Chartered Institute of Public Finance and Accountancy (CIPFA). Jeffrey Matsu and Dr Mehdi Shiva co-lead the sessions.
Session overview
The PLG’s second meeting explored the theme of measurement and economic validity. What are we measuring, how is it defined and when should organisations go about collecting the necessary data? Crucially, we discussed the importance of developing an analytical framework that encompasses the design, monitoring and evaluation of policy throughout its life cycle. This holistic approach was considered both an imperative and challenge to the execution of good policy – and the durability of its outcomes.
This discussion sets the context for our next meeting in March which will focus on innovative pricing methods in the areas of healthcare, social policy and climate change. The idea here will be to consider the multiple dimensions of ‘value’ and how these can be translated into market-friendly pricing frameworks that can be brought to scale.
Speaker Panel
During the session, a mix of policy experts introduced their tools and approaches for appraisal and evaluation.
[Francis Markus, Principal Researcher, Research Team at Greater Manchester Combined Authority] We began the session by exploring cost-benefit analysis (CBA) and measurement of unit cost for supporting decision-making for service and policy redesign. Collating evidence from multiple sources and validating their robustness through collaboration with relevant stakeholders enables comprehensive assessment of fiscal, economic, and social costs and benefits that are kept up-to-date with new research and policy development.
[Vicky Davis, Director for Local Government Value for Money at the National Audit Office] Placing a local government perspective into a wider context, we discussed the importance of national auditing framework in evaluating and improving value for money, defined in terms of economy (spending less), efficiency (spending well), and effectiveness (spending wisely). While evaluation methodologies may vary across policy areas, guiding frameworks and a portfolio approach that cut across departmental boundaries are conducive to accountability and value for money.
- Evaluating Government Spending
- Improving Government's Planning and Spending Framework
- Framework to Review Programmes
- Framework to Review Portfolios
[Paddy Carter, Director, Research and Policy at CDC Group] Extending the national level discussion on value for money, we explored the challenges in measuring impact in development aid and investment. We discussed the use of scoring and ranking of impact defined in terms of economic productivity, sustainability, and inclusivity to strike a balance between diverse objectives. Insofar as the notion of ‘impact’ remains inherently subjective, the use of a score can be a pragmatic way to formalise the institutional view of impact and resolve the trade-offs between competing objectives while enhancing accountability and predictability at a portfolio level.
Peer learning group themes
There are growing public demand to demonstrate the ‘value’ of public expenditure. The tendency is to rely on cost- or price-based approaches in measuring the effect of public programmes. More recently, there has been growing awareness calling for broader measures of value to society – what is known as social- or public-value – utilising new metrics alongside traditional ones in making decisions. This affects economic and financial analyses such as value for money and favours forward-looking approaches such as value- or outcome-based approaches.
The UK’s recently published ‘Sourcing Playbook’ means that procurement decisions are, in theory, subject to scrutiny, ensuring that officials are accountable for the way they spend public money. Covid-19 has demonstrated systemic shortcomings in public service provision across the globe. Meanwhile, cross-sector partnerships between public, private, and the third sector could help societies solve critical problems, while spurring growth, employment and ultimately resulting in positive spill-overs across many sectors. Crucially, sourcing decisions should be made on a case-to-case basis, using past experiences to help inform deliverability, affordability and value for money.
Good governance and regulation, facilitated by standard measurement methods for value and effective public-private partnerships, could benefit from private resources in creating value for the public. This would allow governments to co-create markets by involving private investors (especially those interested in social impact) in activities which yield the highest positive return to society.
A traditional view in economics has been that greater social equity and national efficiency (i.e., growth and productivity) cannot be achieved simultaneously due to a policy ‘trade-off’ between the two. Such thinking has contributed to a surge in spatial and social inequalities across the world including the divide between ‘superstar cities’ and ‘left-behind places’. Such disparities were most recently observed in the human cost of the pandemic as well as the staging of vaccination roll-outs.