Using near real-time data on aid flows: Lessons learnt

With the arrival of the Covid-19 pandemic, Development initiatives (DI) recognised that there was an urgent but unmet need to assess how donors were responding to the crisis month on month. This required disaggregated, near real-time data. Our experience analysing the data available demonstrated once again the need for timely publication of quality data.

DI is well acquainted with the difficulty in getting near real-time data. For almost 30 years we have been number crunching data on humanitarian and development assistance. Our analysts are amongst the most experienced users of the Organisation for Economic Cooperation and Development (OECD)’s Creditor Reporting System (CRS), the arbiter of official development assistance statistics. But the CRS only provides data at two points in the year − top-line provisional aggregates in April, and detailed data at the end of the year.

DI is also very close to the International Aid Transparency Initiative (IATI). For the last 13 years, we have played a leading role in the development of the IATI standard, and are the technical lead within the IATI secretariat. More than 1,200 institutions now publish to the IATI standard.

The benefits and challenges of near real-time data

Until a year ago DI analysts had, for good reasons, been reluctant to use IATI data. Unlike the CRS, IATI is a voluntary standard. As such, despite gradual improvements, the data has never been comprehensive, and data quality has been patchy.

CRS and IATI data cannot be compared like-for-like. The CRS contains manually curated, annual statistics that are a country or institution's mandatory official submission to the OECD. ^[1] IATI data is typically machine-generated from management information systems. What it lacks in formal authority it gains in its timeliness, forward-looking budget projections, transactions disaggregated by time/recipient/sector, and richer activity details including locations, results and documentation. The result being that while regular comparison of a selected basket of IATI publishers does not produce absolute aggregate numbers as the CRS does, it does show trends in portfolio make-up over time. It does so much closer to the moment those decisions are being made and, critically, creates more potential and time to engage and improve policy and decision-making.

Using IATI data to analyse response to the Covid-19 pandemic

Given the greater timeliness of IATI data, and knowing that our IATI team had been working closely with publishers to improve their data, we took another look. We found that 27 institutions, who between them recorded US$143 billion of spending in 2019, were publishing near real-time data of a quality good enough to use. By near real-time we mean activity data with financial transaction values disaggregated monthly (or at most quarterly) that are available two or three months in arrears. A further 18 institutions (with a combined spend of US$55 billion in 2019 ) publish data that, if handled with care, could also be used. ^[2]

In July 2020 we published our first analysis asking How are aid budgets changing due to the Covid-19 crisis? This factsheet compared donor commitments for the first five months of 2020 with the corresponding months in 2019. This work has continued and we now produce monthly updates with interactive visualisations in our data tool: Tracking aid flows in light of the Covid-19 crisis . In June 2021, for example, we published data for the period April 2020−March 2021, and the corresponding period in the previous two years.

What have we learnt from working with IATI data?

Two of the biggest challenges facing users of IATI data were overcome early on in our work. Firstly, IATI encompasses the entire aid delivery chain. The same activity is therefore accounted for by multiple actors with resulting double counting. In an ideal world, the task of cross-checking flows between reporting organisations would be simple. Since that process has inherent complications, this problem is largely overcome by analysing bilaterals, international finance institutions and multilaterals separately as very few funds are disbursed within, rather than between, these groups.

Secondly, we soon discovered that there are differences between the quality of commitment and spend data across a number of large institutions. For example, Germany’s Federal Ministry for Economic Cooperation and Development produces accurate commitment data, but cumulates its spend across years. The United Nations Development Programme, meanwhile, has accurate spend data but cumulated commitment data. DI keeps a publisher reference table to record these anomalies and ensure that misleading data is automatically excluded from our selected baskets. From a near real-time data point of view, ensuring the accuracy of transaction dates is the biggest – and simplest – improvement that a number of large IATI publishers could make to improve the usability of their data.

Our model relies on consistent data over time from each of the selected publishers. Inconsistencies, particularly from large institutions, have the potential to undermine the credibility of this work. When, in April 2021, we came to summarise trends for 2020 as a whole, the discovery that the UK’s Foreign, Commonwealth & Development Office commitment data had become unpredictable and US data (with the exception of data from USAID) had not been updated quarterly as usual brought us close to drawing misleading conclusions on overall bilateral trends. ^[3]

While the basket of publishers that we have been learning to work with is producing results, dangers lie in IATI’s lack of comprehensiveness. In preparation for forthcoming papers on domestic and international resourcing in Kenya, we had noted the increase in World Bank and African Development Bank loans to East Africa. At the same time there were protests on the streets of Nairobi about the government’s capacity to repay its debts. These were sparked, however, by a US$2 billion loan from the International Monetary Fund (IMF). The IMF is not currently publishing using the IATI standard, so we had to scrabble to download this data from the IMF website. Along with the IMF, Japan and South Korea are also lapsed publishers, creating significant holes in our analysis.

The challenge doesn’t end there. Increasingly both the CRS and IATI suffer from the lack of data from China and other emerging actors in the humanitarian and development world. And, with the exception of a few highly fragile states, the dependence of low-income countries on aid is diminishing year-by-year as other cross-border financial flows become increasingly important.

What next?

Where does this leave DI? How do we move forwards in using near real-time data with our credibility as resource flow analysts intact? Do we admit defeat and rely on the CRS’s rear-view mirror? Do we put our eggs in the incomplete, inconsistent IATI basket? Or do we think outside of the established boxes and rephrase the problem that we are trying to solve: How do we access all available data on cross-border resource flows in order to produce evidence that contributes to a better allocation of financial resources?

There are a couple of ways in which DI is looking to keep moving forward with this. We began working on interoperability some years back. Our Development Data Warehouse provides a one stop shop for our analysts to access data from the CRS, IATI, the UN OCHA’s Financial Tracking Service , the World Bank’s World Development Indicators and PovcalNet from a single, common, platform. It also houses our Spotlights on Kenya and Uganda which harvest subnational data from a range of official sources. Plans are in the pipeline for this joined-up service to be shared as a public good.

Most of our analysis of IATI data is done from a transactions table in which values are split according to IATI’s multi-recipient and multi-sector. We will use this table, and IATI’s architecture, as a base for including non-IATI data – harvested by our data scientists from an expanding spectrum of sources -- so that all near real-time transaction data can be accessed from a single source.

Globally, a more joined-up picture of flows could lead to less duplication of effort, better identification of gaps and mutual benefits between private and public investments. At a country level, integrating data on cross-border flows with domestic finances could rationalise and enhance national development planning.