Reflections on the 2022 INDIGO Spring Hack and Learn

Our International Network for Data on Impact and Government Outcomes – INDIGO – is a group of practitioners, researchers, policy makers and data enthusiast who share an interest in using and reusing better data on social outcomes projects. Even if data lies at the core of our INDIGO initiative, we are convinced that data alone is not enough. Bringing together a community of peers who can make sense of our various datasets and translate them into actionable insights is also a key part of our mission.

To this end, in addition to our usual INDIGO peer learning events, we host biannual Hack and Learn events. They are great opportunities to engage with the community of practitioners, meet new colleagues and address a series of challenges together. Our Hack and Learn is different from other hackathons because our focus is on the learnings and lessons that participants create in these two weeks of work, rather than creating complex data products.

After a kick-off session where leaders pitched their challenges, participants selected challenges to work with. They joined their corresponding Slack channels and had two weeks to address their challenge and prepare a presentation on their learnings. We all came together at the final Show-and-Tell session, where participants showed their results and a panel of experts - Silvana Fumega and Carolina Pedrosa Gomes de Melo - provided feedback on the final products.

For this edition, participants could choose from five challenges. The aim of this blog is to summarise the main lessons and insights generated by each team. With this goal in mind, we invited one participant from each team to describe their experience and share future plans.

SDGs and outcome metrics (Jorge Norio Rezende Ikawa, Brazil)

The SDGs and outcome metrics challenge brought together two fascinating topics: social impact bonds and the United Nations Sustainable Development Goals (SDGs). As a start, we had the INDIGO Impact Bond Dataset with the contract’s characteristics and social outcomes metrics. Our goal was to link every outcome metric to one (or more) related SDG goal and target(s). This exercise had three main objectives. First, we aimed to add one more data point into the dataset linking outcomes to the corresponding SDG. We also wanted to generate a better visualization of the data through a Sankey diagram, so we could see the connection of the social impact bonds to different SDGs. Finally, once the Sankey diagram was built, we analysed the graphic to understand whether existing contracts meet the most urgent demands.

Our final product can be seen here. Previous hackteams have done the same data entry work for different subsets of projects and we have all built this interactive Sankey diagram. We look forward to future Hack and Learn challenges where we can add data on SDG alignment for the rest of the impact bonds in the dataset. Once we have data for all outcome metrics, we would like to establish links between SDGs and places (using delivery location data) to better understand if these projects are addressing the most urgent demands of their respective regions.

SPARKS challenge (Richard Dooner, Wales, UK)

The Strategic Procurement, Applied Research and Knowledge Sharing (SPARKS) challenge focused on how public contracts are being used to pursue economic, social, and environmental goals beyond the core goods, works or services being procured. This strategic use of public procurement is called different things around the world, including sustainable procurement, buying social, social clauses, community wealth building, broader outcomes and social value.

The SPARKS team decided to build a spreadsheet of government outcomes frameworks and strategic public procurement rules and/or policies. The aim of the group was to gather examples of outcomes frameworks (e.g. Scotland’s National Performance Framework) and strategic procurement policies across the world. At the end of the event, the spreadsheet had data on outcomes frameworks from Wales, Scotland and Northern Ireland (United Kingdom), South Africa and New South Wales (Australia).

Although more work needs to be done to collect data on outcomes frameworks from other parts of the world, we hope to use this information to explore the effectiveness of different approaches to social and environmental outcomes in different contexts. From our initial analysis, it was evident that there were public policies in common and that the priorities of different municipalities, and indeed nations, were linked by a desire to improve social outcomes by leveraging of public expenditure. It was a privilege to help and to share some of the lessons learned here in Wales, and I look forward to seeing the next steps of this group and learning from other colleagues.

Philanthropic giving in India (Honey Agarwal and Divya Chopra, India)

The Centre on Social Impact and Philanthropy (CSIP) has compiled publicly available data on various forms of philanthropic giving in India. For this event, CSIP shared the dataset on foreign philanthropy and asked the challengers to analyse the dataset and identify any trends in foreign funding for the last three years.

The team analysed data from both the recipient and donor sides. On the recipient side, the team found out that the six southern Indian states bagged more than two-thirds of the total foreign funding. On the donor side, the team created a visualization for the share of a country’s funding that is provided by institutional donors. Countries were divided in quintiles, according to their GDP level. We explored how the share of funding from institutional donors varied across time for different quintiles and came to the conclusion that countries in lower quintiles had higher variation.

These insights are extremely useful in the context of a developing country like India, because foreign philanthropic funds aid the process of development and directly impact the lives of people. Going forward, the team recommended merging the latest data (post-2019) and analysing the impact that Covid-19 had on donor and recipient trends.

The Village Enterprise challenge (Ozioma Paul, Nigeria/UK)

The Village Enterprise DIB is the first-ever Development Impact Bond for poverty alleviation in sub-Saharan Africa. The primary goal was to improve the income levels of at least 12,600 extremely poor households in rural Kenya and Uganda.

For this challenge, the Village Enterprise and IDinsight team kindly shared data from the randomized controlled trial (RCT) evaluation of the programme. We first looked at the dataset and realised that data was large and complex. We also had access to the recently published report and, as we did not want to reinvent the wheel, we studied it to see where we could make a new contribution in the little time we had.

We came up with some simple visualizations that would help a policymaker to get a rapid understanding of the programme’s impact. This included two dot plot graphics, where we could see the differences between averages in income and assets from control and treatment groups.

We also discussed the usefulness of averages. Although averages are extremely helpful as a summary figure, they may be hiding internal disparities among a group. This is why we decided to plot distribution curves to understand how different households were achieving growth in terms of income and assets.

The dataset was rich and full of other research opportunities. Areas for future exploration include reusing data on consumption to better understand what families are consuming, and how this indicates improvement in their financial status. Future hackteams could do more work to analyse regional disparities across Kenya and Uganda.

The Education Outcomes Fund challenge (Naruetaya Pongcharoen, Thailand)

The last challenge was about designing a data platform for the Education Outcomes Fund (EOF). A group gathered to design the digital forms that service providers, evaluators, and investors working with EOF projects would have to report on. A second piece of work included the design of the general overview of the data platform for EOF members.

We divided our groups based on individual interests and past experiences: there was a service provider subteam, investor subteam and evaluator subteam. Every subgroup met weekly to deliberate best practices regarding data collection and create the forms. We reviewed other groups’ designs to make the forms coherent during the second week, before the final presentation. Our work was relevant for the field because a comprehensive data platform means practitioners can collect data and collaborate more efficiently. The literature indicates that evidence and robust monitoring system are keys to the success of social outcomes contracts. For this reason, we look forward to seeing the progress of the EOF projects in Sierra Leone and Ghana, and will keep an eye on the data and learnings coming out of this initiative.

Next steps

We had a great time working together during the two weeks, and are very grateful to all the participants who dedicated time and effort to their challenges, and to the external partners who co-hosted the event (INSPER Metricis and the Centre on Social Impact and Philanthropy from Ashoka University) or ran particular challenges (Village Enterprise and the Education Outcomes Fund). We learnt about social outcomes contracts, SDGs, foreign philanthropy, outcomes fund and development impact bonds; and we have pressing questions and challenges that we still want to work on, such as completing our Impact Bond Dataset with data on SDG alignment, understanding the impact of the pandemic on foreign philanthropy funds in India or discussing how best to present complex data for non-technical audiences.

The next Hack and Learn event will be in September 2022, right after our Social Outcomes Conference. We are looking forward to the next opportunity to get together with the INDIGO community and address a different set of challenges.