Real-time evaluation

What real-time evaluation can do for you

Imagine this:

Business as usual: The NGO Eduasusual mobilizes funding for keeping girls in school. They put together a package of interventions (school feeding, awareness raising, teacher training, etc.) and start implementing it. The package is the same for each of the 150 schools, and roughly based on a “Theory of Change” designed by an external consultant who flew in and drafted the project document. At the end of the intervention, the organization commissions an external evaluation to understand if the approach has worked or not. So far, so boring.

And that is where outcome real-time monitoring is coming in: Instead of documenting school attendance in a paper-based registry, target schools use a simple SMS-based system (like UNICEF’s EduTrac). School attendance is tracked on a daily basis and analysed on an online platform.

This allows the NGO to quickly try out what works – and what does not – on the outcomelevel (!). Together with a group of pupils, headmasters and teachers, they put together three different packages of intervention. Each package is implemented in 50 schools. Over the next three months, they closely follows what happens. After a while, they identify the set of interventions that was able to rise school attendance. The analysis is based on the attendance data collected as well as an internal review with the students, teachers and headmasters involved at the beginning. With this “proof of concept”, they mobilize more resource, scale up the successful intervention to all 150 schools. With additional funding, they put again together two variations of the successful model and try it out in 100 additional schools.

Is there a problem with ethics? No, not really. The NGO does not withhold any benefits from some schools – they simply do not know yet what works and what does not. Covering all schools in the country from the beginning is not possible anyway with limited funds.

Donors could use a similar approach: They fund five NGOs with the simple goal to keep more girls from skipping school. How they do that is completely up to them: no logframes, no detailed planning of activities, no detailed reporting. ButDonors continuously track how well the NGOs perform in keeping girls in school. After 6 months, they review the data and scale up what worked.

What real-time evaluation can do for you

by Thomas Winderl time to read: 2 min