As time and resources are often in short supply, it is good to openly discuss the challenges that arise when collecting data. This way, you can be clear about the implications of your choices in the design of your evaluation (adapted from PMERL):

• Using existing data versus obtaining new data: There will always be a balance between what you would like to monitor and evaluate and what is actually possible given the time and resources available. Using data that already exist and are easily accessible makes sense, especially when resources are limited, but these data may not be the most relevant and may oversimplify or even distract from the overall aim of building resilience. For example, existing data might give you an average measure of soil moisture content at the field level, but when looking at plant scale impacts, this is no substitute for smaller-scale moisture measurements where the sampling takes into account changes in soil type and field topography.
• Identifying locally appropriate indicators versus externally determined indicators: Good adaptation is locally specific and M&E systems need to be tailored to local conditions. However, engaging farmers and other stakeholders in the development of these indicators can be time-consuming, which make externally determined measures seem more attractive. External measures may also be easier to compare with other areas. One way to address this is to interpret externally derived indicators to ensure they are relevant at the local level. For example, an externally determined indicator might be access to accurate climate data. This could be locally interpreted by asking if there are good links to meteorological stations and research organizations.
• Building capacity to do M&E versus using external experts: M&E can be used as an opportunity to empower coffee farmers and other stakeholders to learn systematically from their experiences. To build long-term capacity, farmers need to not only participate in adaptation processes, but also to design and manage these processes themselves. This requires a greater investment of time, as this type of change does not happen quickly. Outsiders can perhaps do the work more quickly, but it would build far less capacity for adaptation in the long term. It would be ideal if local farmers could be trained to do the evaluation themselves, which would build their capacity to identify the severity of climate risks, identify assumptions about what activities would build local resilience, and develop a plan to gather evidence in testing these assumptions.
• Evaluating the success of planned activities versus learning from the unanticipated consequences of the work: The evidence needed for M&E is often a mixture of easily measurable data relating to the achievement of activities and more qualitative ‘stories of change’ that can reveal things that were not anticipated in the beginning. These stories of change are important, as they help to challenge assumptions about what supports good practice and what gets in the way.

An indicator provides specific information on the state or condition of something. In M&E, it is often about providing information about change (e.g. have farmers become more resilient?). Indicators are an important part of understanding change processes and exploring what adaptation measures work or do not work, in what context and why.

There is not only one set of indicators that will work for all adaptation implementation processes. Indicators must be chosen in relation to the adaptation activities that were planned and the context in which these activities were implemented. By developing indicators as part of the project pathway, you will ensure that they relate to your objectives.

Assuming you developed a project pathway in Step 3, this task will help you to gather the information in order to understand how the pathway works in practice. If you have not developed a project pathway, this task will still help you gather the evidence you need for your M&E activities, but it is recommended to take a look at Step 3 first.

Indicators are present in both monitoring and evaluation processes, but not all indicators will be used for both. For example, it may be expensive and logistically impossible to track the attitudes of farmers throughout the implementation of adaptation options (as part of your monitoring), but you may wish to do so as part of a mid-term evaluation. Similarly, your evaluation may not require monthly data from demo plots, but will instead use summary data on how the demo plots performed overall. Monitoring progress is reliant on selecting indicators that are capable of representing changes. These indicators should link to your efforts to implement and validate adaptation options (e.g. make use of observational data from test plots, see Table 19).

There are two basic types of indicators for M&E and most processes are likely to be a mix of both:

Outcome indicators demonstrate that a particular outcome has been achieved (e.g. reduction in disease-related economic losses amongst smallholder farmers). Outcome indicators are very useful, but can often be difficult to use in assessing adaptation activities as there are often long time lags between the implementation of the adaptation option and the outcome being achieved (e.g. if there is no rust outbreak in an area, how can we know if losses are reduced as a result of the project?). It is thus useful to also use process indicators to measure progress towards the achievement of an outcome (e.g. the number of farmers now using coffee rust prevention measures or the number of farmers who have been trained). These process indicators are valuable in understanding whether resilience is increasing, even if the resilience has not yet been tested by a climate-related event.

To choose which evidence to collect (or which indicators to measure), look at the evaluation questions you developed in the previous section and consider for each what type of evidence or indicator is most suitable, given the resources and capacities available. Record the key outputs from this task in the M&E plan template, Step 5, Task A.