This task is about exploring and defining what are you hoping to achieve from the M&E process, what you are trying to monitor and evaluate and whom you need to involve. Addressing these three issues will allow you to identify evaluation questions and develop a plan to collect information and analyze it in a useful way.
What is your motivation for undertaking an evaluation at this time? | Further considerations: What aspects of the project do you want to know more about? Who are the audiences for the evaluation and what are their needs? |
Do you need to demonstrate to others that you have done what you said you would do? |
|
Do you need to demonstrate how successful the adaptation activities were? |
|
Do you want to share what is working well and what supports this? |
|
Do you want to improve decision-making processes? |
|
Do you want findings to act as a guide to future work? |
|
Do you want to motivate other people to act? |
|
It is important to be clear on why you want to undertake M&E activities, as this will shape how they are designed, who becomes involved and the evidence you choose to gather.
Reasons to undertake M&E:
In addition to assessing the technical performance of adaptation measures (e.g. is a rainwater harvesting system gathering water effectively?), M&E also examines the outcome of these measures and how well they worked in different situations.
For a rainwater harvesting system, this might include comparing the efficiency of this system to other available options, measuring cost-effectiveness, determining who benefits from the system (and who does not) and establishing whether the system has any negative consequences (e.g. encouraging the use of excess water).
When identifying a purpose, it is also important to consider when you intend to undertake the evaluation aspect of your M&E approach. For example, if you are conducting a mid-term evaluation, an important purpose might be to understand how aspects of the project might be improved or enhanced in the short term.
Ensure that you discuss the purpose of your M&E process with key stakeholders. If you explain why the evaluation is taking place, and initiate a dialogue on the benefits of the M&E outputs (including the benefits for them), stakeholders are likely to engage with the process in a more positive way.
It is also important to consider what exactly you will be monitoring and evaluating. There are a huge number of possible adaptation options and approaches in response to a range and combination of possible impacts (heat, rainfall, disease, wind speed, etc.). You may also be monitoring and evaluating a single adaptation option or a whole project or program. However, given the objectives of c&c it is reasonable to assume that you will need to monitor and evaluate:
These broad themes will require further refinement. Setting clear boundaries for what you are going to evaluate will help you select the most appropriate methodology. By carefully considering what you are monitoring and evaluating, and linking this to the objectives outlined in your project pathway, you should have a strong foundation on which to develop a) appropriate indicators to monitor and b) a strong set of evaluation questions.
Guiding questions for defining what you are monitoring and evaluating:
Deciding who will take part in the M&E process and the roles they will play requires a balance between including everyone with useful knowledge and experience, and managing what is practical in terms of the available time and resources.
Guiding questions for planning whom to involve in M&E:
As a minimum, key people such as farmers and extensionists should contribute by providing information and experience. However, more participatory approaches to monitoring and evaluation can be especially useful. These require more active involvement of farmers and other stakeholders in the development and design of the M&E process. They would also play a key role in discussions on how the success of the adaptation process is defined, the evidence that is needed and data collection and analysis. The role of the person or team responsible for implementing M&E processes can thus shift from full control of the whole process to a facilitation role for others.
A ‘good’ facilitator serves mainly as a catalyst or stimulator, rather than a leader, drawing out and bringing together inputs from different types of stakeholders. This requires key skills of negotiation and in some cases conflict resolution. Facilitators should ask the right questions at the right time listen well, build trust, encourage the sharing of ideas, and at the same time keep the group focused.
Being clear about how you decide who will participate will help people understand what is expected of them. Bringing coffee farmers and other local stakeholders together to evaluate and learn from the work requires methods that appreciate the value of different perspectives and different types of evidence (e.g. opinions, experiences, factual data and cultural values). Some participants may need additional support (e.g. time, guidance, access to data) to enable them to contribute effectively.
A facilitator can be used to build the M&E team’s capacity to engage people, support fair participation and encourage locals to take an active role in managing the M&E process. If there are significant power differences between different participant groups, it may be worth meeting each group on a one-on-one basis before bringing them together. This will allow them to consider their individual perspective before sharing it with others.
Also refer back to the stakeholder mapping created in Step 2. Many of the stakeholders identified and engaged during the assessment and adaptation planning stages may also need to be engaged in the M&E process. Reflect on how these stakeholders have been involved to date and consider how they might contribute to, or benefit from, the M&E process.
Participation | Design your M&E to include those most affected by the work being undertaken. |
Negotiation | Encourage open discussion about what will be monitored and evaluated. It should not just be based on the views of the most influential people. |
Learning | Everyone involved in the evaluation process should be open to learning and should be supported to do so through access to information, sharing of experiences and facilitation of reflection to think more deeply about the practical implications of these findings. |
Flexibility | Allow plans to change over time to incorporate new learning and understanding. |
The following section will provide suggestions for exercises that will help you to develop and implement an M&E plan, which was introduced in Step 5. You are not required to use every exercise suggested here or to follow any particular sequence, but rather to choose what is appropriate for you – exercises work better for some groups than others.
The overall purpose of the c&c initiative is to build the resilience of the local coffee sector to climate change. However, it is important to contextualize this broader purpose for the local level and to think through the attributes that a resilient local coffee production system might have (see introduction in Section 1). Use the following table as a template to record the outcomes of each of these steps. Once you have completed each task and filled in each section of the template, you will have a full M&E plan.
As required to understand the purpose and given the resources available.
Stakeholder action plan | |
1 | Define a purpose What do you want to achieve in the evaluation process? You can then decide who needs to be involved in order to achieve this objective (refer to part A of the M&E plan). |
2 | Identify stakeholders Who needs to be involved? Refer to previous work on stakeholders and any outputs from past exercises, e.g. brainstorms, Venn diagrams, influence and importance matrixes. Establish:
Also ask: How much do larger factors (e.g. institutions, markets or governments) affect what can be achieved at the farm level? What are the implications of this on stakeholder involvement? |
3 | Establish roles Who takes on which responsibilities? Roles in M&E include change throughout the process e.g. :
|
4 | Understand stakeholders What do you already know about these stakeholders? What can already be figured out from previous discussions about the stakeholders, including their perceptions, knowledge, interests, decisions, power differences, and patterns of interaction among themselves and with others? |
5 | Select stakeholders Which individuals can act as good representatives for their organizations or communities in the M&E process? Once you have identified relevant categories of stakeholders, you now need to get the names of appropriate people. |
6 | Look at logistics When is the best time to approach these stakeholders? Is it best to involve them at the beginning, at a later stage or throughout? In which season, on which day of the week or at what time of day will you approach them? Where is the best place to meet stakeholders? Will you meet them separately or in shared meetings? In a community building, a public space, at home or at their place of work? |
7 | Anticipate problems What problems can already be identified? Do those involved (scientists, farmers, advisors, funders, etc.) value different types of information? If not, how can this be managed? How can you support the participation of poor coffee farmers in an evaluation process that might feel quite unfamiliar to them? Will participants change over time? How will this be managed? |