Categories
Indicators

Indicators in the age of Open Data

In the age of open data, indicators must be clear, credible and proportional.

It is a game changer for M&E: Communication skills using indicators suddenly become a core qualification.

More and more organizations make their data on aid, development and humanitarian flows accessible online as a result of the International Aid Transparency Initiative (IATI). This year, large donors like the UNDP even add project indicators, baselines, targets and status data available online. This is a very good reason to look hard at the indicators we are using. Anyone on this planet with internet access can look at our data. The times when M&E was seen as a highly technical speciality are gone. Development lingo, awkward formulations and technical expressions do not work anymore in the age of open data. M&E specialists now – also – need to be excellent communicators.

That is why indicators in the age of open data need to be – apart from technically sound:

CLEAR

The user of our indicator data  – clients, journalists, academics, donors – data need to understand what we say. We should use clearer language, avoid technical expressions or abbreviations, and add background information so that the indicator can be understood if looked at in isolation.

CREDIBLE

There will be a much tougher scrutiny of our indicator data. While indicators in the past, let’s be honest, were looked at by very few people directly involved in an intervention, a much larger group of people – some of them very critical of a programme – will look very carefully at our data. And since the net never forgets, our data will be stored basically forever. This requires us to be much more certain that we can back up all data with rock-solid, credible evidence.

PROPORTIONAL

When linking up indicator data with the equally publicly accessible budget and expenditure reports, anyone with a calculator can do rough value-for-money calculations: This project raised 5000 people out of poverty, but it did cost 10.000 USD per person. That intervention had 700 schools built for 25.000 USD each. These calculations are valuable, but we need to ensure that the indicators we pick properly capture the key outputs or outcomes of an intervention. In short: Indicators and data need to be proportional to the funds used.

Categories
United Nations

UN Development Assistance Frameworks, brand new ;-)

As you will know, the new guidance for UN Development Assistance Frameworks (UNDAF) is here! Web-based with nice layout at https://undg.org/programme/undaf-guidance/

Not sure what you think about the new UNDAF guidelines, but here are some of my thoughts after an initial review:

Despite the positive change in how the guidelines, annexes and companion guidance are presented, this appears more of an evolutionary update rather than a ‘SDG revolution’. True, the guidelines include a clearer focus on the SDGs (good!), but I do not really see any radical departure from how the UN has done business in the past. Links to the SDGs in the results framework remain tentative at best. The new UNDAF guidelines have clearly taken on some of the features of One UN, but most of the prescriptive, practical guidance remains similar to the 2014 UNDAF updates.

Having said that, there are some really positive changes in the new guidelines, in my view. For example:

  • The central role of Leave No One Behind for everything the UN does, which – at last – provides again an overall vision for the entire UN. If taken seriously, this could become a very powerful mobilizer, internally as well as externally.
  • The inclusion of a strong theory of change: although a buzzword and very much en vogue, it can indeed help – if properly done – to focus the UN’s work and provide a common framework for working together
  • It is a good thing that the Common Country Analysis is now a minimum requirement: This is important since solid theories of change (and the subsequent results chains) require a detailed, logical problem analysis as a first step. Otherwise, they tend to fail.
  • It is positive that the UNDG RBM handbook is included as a pillar for the UN’s work. Although not perfect, the RBM handbook provides a common, reasonably clear set of definitions, frameworks and tools to all UN agencies.
  • Another step in the right direction is that UNDAF data should, where possible, be publicly accessible (in line with IATA).

However, I feel that the updated UNDAF guidance overall falls short on a number of issues which would have shown that business as usual is not an option. I was particularly surprised by the lack of attention to innovation. Although there is some reference to real time data collection and vague and general support of innovation, this is certainly not the big push to ingrain innovation into the DNA of the UN and to catch up with some of the more advanced techniques for innovation (positive deviance, using big data, testing, Minimum Viable Products, etc). I would especially have expected a stronger push for adaptive programming, which appears to quickly become mainstream in development organizations.

So overall, my personal feeling – after a first read – is that the new guidelines are not as different from the 2014 version as I would have expected and the nice, modern lay-out initially suggests: The guidelines pick up some of the One UN elements (which is good!), provided a bit more clarify and simplified a few things, but overall, my impressions is that resulting new UNDAFs will not look radically different from what we know today – and that the way the UN will cooperate/coordinate at the country office level may not dramatically change.

Has anyone tried to apply the new guidance to an UNDAF formulation? Would love to hear your experiences.