Categories
Indicators Planning for M&E

Indicators: choose the right angle

Once we have agreed on a clever, limited set of indicators, all seems good. However, we are not yet there. For most indicators, we can choose a specific ‘angle’.

What do I mean?

For example, we are considering the “Number of female members of Parliament” as an indicator. Which may be fine. However, we may prefer the “Proportion of parliamentarians that are women” instead (and often is preferable). On the other hand, we may want to compare it with the % of voters that are women. Or we want to capture the change over time – or even the funds invested in political leadership by women.

Soooo many options. But we need to get the angle right.

Categories
Monitoring and Evaluation

Monitoring? Evaluation? Isn’t it the same?

Don’t be fooled. 

The fact that the terms Monitoring and Evaluation often goes together and is called “M&E” is somewhat misleading.

When governments or development organisations use the term “Monitoring & Evaluation”, however, they mean something very specific: Monitoring & Evaluation is about collecting and analysing data and reporting on findings on how well a programme, a policy, a service or an organisation is performing, and making a judgement about its value.

It is true that Monitoring & Evaluation frequently share similar tools and methods. While they are interrelated, Monitoring & Evaluation are clearly separate activities.

So no, it’s NOT the same – it’s actually quite different. Monitoring and Evaluations are usually carried out by different people and differ in how often they are carried out.

Ok. So what is monitoring?

In a nutshell: Monitoring is like the dashboard of your car when you are driving: It tells you have fast you go, how much petrol you have left, or maybe if one of the car’s door has been left open.

In governments and development organisations, monitoring is he regular and systematic collectionanalysisreporting and use of information about programmes, policies or services.

Monitoring is concerned with the performance of a programme, a policy or a service. Unlike an evaluation, it is typically conducted internally. That means monitoring is typically carried out by staff that works inside an organisation. And unlike evaluations, it is a continuous process. That means it is carried out non-stop during – and sometimes after – an activity. Monitoring typically supports the management of programmes, policies or services, and helps to manage its risks.

…and what is evaluation?

Evaluations are like the occasional check-up of your car: Evaluations are a systematic and impartial assessment of expected and achieved accomplishments.

Evaluations take a step back to look – as the term suggests – at the overall value of a programme, a policy or a service. Evaluations are usually conducted externally. That means evaluations are typically carried out by evaluators or specialists with no link to a programme, policy, service or organisation. Having independent, external evaluators should insure a more unbiased judgement. Unlike monitoring, an evaluation is not carried out all the time, but is a one-off activity. Typically, evaluations are carried out during or at the end of an activity.

And evaluations are more systematic than monitoring: Here are some typical questions an evaluation attempts to answer:

  • Is a programme, a policy, a service or an organization relevant? Does it suit the priorities and policies of the target group?
  • Is it effective? Does it achieve results?
  • Is it efficient? Does it achieve results at reasonable costs?
  • Does it have impact? What real difference has a programme, a policy or a service made to for beneficiaries?
  • Is it sustainable? Will positive changes continue once funding is cut?

That is why evaluations tend to be broader in scope then monitoring.

Categories
Innovation Monitoring and Evaluation

M&E needs to change. We need to change.

Monitoring and Evaluation is an exciting area of work because it is currently undergoing rapid changes.

Countries and organizations are increasingly taking innovative approaches to monitor and evaluate the performance of programmes, policies, services or organizations.

There are several reasons why this is happening.

‘RESULTS’ ARE HERE TO STAY

First, citizens, parliaments and donors are rightly demanding to know what results are being achieved with their money. Today, it has become unacceptable to simply report back on the number of people that were trained. Instead, Monitoring and Evaluation is challenged to provide meaningful information about real changes as the result of a policy, a programme or a service.

INCREASING COMPLEXITY AND SPEED

Second, we live in an increasingly complex and fast-changing environment. We need Monitoring and Evaluation that can capture complex cause and effects and provide quick, real-time feedback to tweak or change a programme, a policy or a service.

ACCELERATING TECHNOLOGY

Third, technology keeps accelerating at an increasing speed. This opens a wide range of opportunities for innovations in Monitoring and Evaluation. We are just only beginning to explore the opportunities of ‘big data’, the ubiquitous use of smart phones, the use of satellite images and drones.

Categories
Planning for M&E Results Based Management

The result chain: a beginner’s guide

Monitoring and Evaluation is about measuring and tracking results. That is why it is important to understand what results are, and how to distinguish between different levels of the results chain.

In general, a “result” is something that happens or exists because of something else that has happened:

  • the results of a football game
  • the final value of a mathematical calculation, or
  • the outcomes of an election.

In development and governance, we use a more nuanced understanding of different types of ‘result’: the so-called result chain.

The result chain distinguished between five logically connected elements:

  • inputs
  • activities
  • outputs
  • outcomes, and
  • impact

A REAL-LIFE EXAMPLE

It’s not complicated – and we use this logic all the time. Let’s take a simple example:

I want to do something about living a healthier life. This is the desired impact. To do that, I want to reduce my weight. This is my planned outcome. To reduce my overall weight, I plan on eating ore vegetables and exercise regularly. These are my planned outputs. Eating healthier requires more conscious shopping habits. More exercise requires me to go running or join a gym. These are some of my planned activities. These activities require some extra time and money. These are the inputs.

ELEMENTS OF A RESULTS CHAIN

Let us look more in detail at a typical results chain of a development or government programme, policy or service. This time we start at the lower end of the result chain and work our way up:

INPUTS

Any programme, policy or service requires resources of some kind. We call these resources inputs.

For example: to put together this course, it took me time to record this video; you need an internet connection and a computer to watch it.

Typically, inputs refer to money, staff time, materials and equipment, transport costs, infrastructure, etc.

ACTIVITY

These inputs are required to carry out a number of activities.

For example: you watch videos of this lesson; you do a quiz; you do some additional reading; you watch the next videos; etc.

So: Activities are actions taken that use inputs to produce higher level results – ‘outputs.’

Typical activities in governance and development are the drafting of a policy document for a ministry, the organization of a media outreach campaign, the training of midwives in a new approach, etc.

OUTPUT

The next level, outputs are typically the result of several completed activities.

For example: after reading this blog post, you have the knowledge to critically review and design your own results chain.

In development and governance, an output is delivered if a group of people or an organization has improved capacitiesabilitiesskillssystemspolicies or if something is created, built or repaired.

Outputs are the direct result of a set of activities and delivered during the implementation of a programme, a policy or a service. Outputs are different from the next level of results – outcomes – because you largely have control over delivering outputs.

That means that if we – and our partners – have the resources and the time to deliver a certain output, we can largely guarantee that the output will be delivered.

That also means, that in turn, we are fully responsible for delivering an output.

Typical outputs are a draft policy document for a ministry, a media outreach campaign, improved skills for, etc.

OUTCOME

Now, this is very different from the next level of results: An outcome is something we hope to achieve as a result of what we do.

In development and governance, an outcome implies that institutions or people do

  1. a) something differently (behavioural change) or
  2. b) something better (change in performance)

The difference of an outcome is that – unlike outputs which we largely control – we can only influence the achievement of an outcome, but it ultimately goes beyond our control.

Typical outcomes are a parliament passing a new law, people changing their behaviour because of a media campaign, midwifes that apply new skills in their daily routine, etc.

Outcomes are typically achieved at the end or even after a programme, policy or service has been implemented.

IMPACT

Finally, outcomes should contribute to a broader impact.

An impact is the long-term effect of programmes, policies or services. It implies a detectable improvement in people’s lives.

Impact typically relates to positive economic, social, cultural, institutional, environmental, technologicalchanges in the lives of a targeted population. An impact is often related to broad national goals or international aspirations like the Sustainable Development Goals. Impact is typically much broader than a programme, policy or service. And: an impact is typically detectable only after a few months or even years.

What about ‘results’?

So: Which of these elements of a result chain do we considered ‘results’?

We usually define ‘results’ – in the context of governance and development – as the top three elements of the result chain: outputs, outcomes and impact. And most importantly, results are not inputs or activities.

Example: The Land of Smokistan

Let us look at a few simplified but typical examples from the world of development and governance:

The country of Smokistan wants to reduce smoking.

The desired impact is that less people die of smoke-related illnesses within five years.

The planned outcome is that fewer people smoke fewer cigarettes.

Smokistan plans to achieve that by increasing taxes on cigarettes and making smoking more difficult in public spaces – these are the outputs.

This requires several activities: for example drafting and passing a new anti-smoking law or funding and implementing smoke-free public zones, etc.

These activities require additional time and money – the inputs.

Example: Domestic violence

Let’s take another example:

The same country aims at fewer people experiencing violence in a domestic setting within three years – the desired impact.

The planned outcome is that women and men openly discuss domestic violence.

The country aims at achieving that by drastically increasing public discussions on domestic violence in social media and traditional media outlets – the outputs.

This requires setting up – for example – a social media unit in a ministry and training influential bloggers and journalists in properly reporting on domestic violence – the activities.

Again, these activities require time and money – the inputs.


Did you like this post? If you are interested, check out my mini-course on Monitoring and Evaluation on Udemy, a learning platform: