7.2 Defining the context

UNDP evaluations support the UNDP human development focus “to help people build a better life” by generating knowledge about what works, why and under what circumstances. Therefore, quality evaluations not only focus on the attainment of outputs and outcomes but also asses how initiatives adapt to the contexts in which they operate and how and why they contribute to outputs and outcomes.

Evaluations must be conceived and designed with a thorough understanding of the initiative and the context within which it operates. The UNDP commissioning unit and relevant stakeholders who are engaged in drafting of ToRs (see Annex 3) are responsible for articulating necessary information for evaluators to have a good understanding of the initiative, the evaluation context, focus and purpose of the evaluation, and key questions to be addressed in the evaluation.

Understanding the initiative 

To produce credible information that will be useful for decision makers, evaluations must be designed with a clear understanding of the initiative, how it operates, how it was intended to operate, why it operates the way it does and the results that it produces. It is not enough to know what worked and what did not work (that is, whether intended outcomes or outputs were achieved or not). To inform action, evaluations must provide credible information about why an initiative produced the results that it did and identify what factors contributed to the results (both positive and negative). Understanding exactly what was implemented and why provides the basis for understanding the relevance or meaning of project or programme results.

Therefore, evaluations should be built on a thorough understanding of the initiative that is being evaluated, including the expected results chain (inputs, outputs and intended outcomes), its implementation strategy, its coverage, and the key assumptions and risks underlying the Results map or Theory of Change. The questions outlined in Table 27 should be understood by the evaluators in conducting the evaluation.

Table 27. Key aspects of the initiative

Key Aspect

Questions to Ask

Demand

What is the need or demand for the initiative? What problem or development opportunity is the initiative intended to address?

Beneficiaries

Who are the beneficiaries or targets of the initiative? Who are the individuals, groups or organizations, whether targeted or not, that benefit directly or indirectly from the development initiative?

Scope

What is the scope of the initiative in terms of geographic boundaries and number of intended beneficiaries?

Outputs and Outcomes

What changes (outcomes) or tangible products and services (outputs) are anticipated as a result of the initiative?  What must the project, programme or strategy accomplish to be considered successful? How do the intended outcomes link to national priorities, UNDAF priorities and corporate Strategic Plan goals?

Activities

What activities, strategies or actions, both planned and unplanned, does the programme take to effect change? 

Theory of Change or Results/Outcome Model

What are the underlying rationales and assumptions or theory that defines the relationships or chain of results that lead initiative strategies to intended outcomes? What are the assumptions, factors or risks inherent in the design that may influence whether the initiative succeeds or fails?

Resources

What time, talent, technology, information and financial resources are allocated to the effort?

Stakeholders and Partnership Strategy

Who are the major actors and partners involved in the programme or project with a vested interest? What are their roles, participation and contributions—including financial resources, in-kind contributions, leadership and advocacy—including UN organizations and others? How was the partnership strategy devised?  How does it operate?

Phase of Implementation

How mature is the project or programme, that is, at what stage or year is the implementation? Is the implementation within the planned course of the initiative? Is the programme mainly engaged in planning or implementation activities?

Modifications from Original Design

What, if any, changes in the plans and strategies of the initiative have occurred over time? What are the potential implications for the achievement of intended results?

Evaluability

Can the project or programme as it is defined be evaluated credibly? Are intended results (outputs, outcomes) adequately defined, appropriate and stated in measurable terms, and are the results verifiable? Are monitoring and evaluation systems that will provide valid and reliable data in place?

Cross-cutting Issues

To what extent are key cross-cutting issues and UN values been mainstreamed and addressed in the design, implementation and results?

 
The evaluation context

The evaluation context concerns two interrelated sets of factors that have bearing on the accuracy, credibility and usefulness of evaluation results:48

  • Social, political, economic, demographic and institutional factors, both internal and external, that have bearing on how and why the initiative produces the results (positive and negative) that it does and the sustainability of results.
  • Social, political, economic, demographic and institutional factors within the environment and time-frame of the evaluation that affect the accuracy, impartiality and credibility of the evaluation results.

Examining the internal and external factors within which a development initiative operates helps explain why the initiative has been implemented the way it has and why certain outputs or outcomes have been achieved and others have not. Assessing the initiative context may also point to factors that impede the attainment of anticipated outputs or outcomes, or make it difficult to measure the attainment of intended outputs or outcomes or the contribution of outputs to outcomes. In addition, understanding the political, cultural and institutional setting of the evaluation can provide essential clues for how best to design and conduct the evaluation to ensure the impartiality, credibility and usefulness of evaluation results.

Box 38. Guiding questions for defining the context49

  • What is the operating environment around the project or programme?
  • How might factors such as history, geography, politics, social and economic conditions, secular trends and efforts of related or competing organizations affect implementation of the initiative strategy, its outputs or outcomes?
  • How might the context within which the evaluation is being conducted (for example, cultural mores, language, institutional setting, community perceptions, etc.) affect the evaluation?
  • How does the project or programme collaborate and coordinate with other projects and organizations? 
  • How is the programme funded? Is the funding adequate? Does the project or programme have finances secured for the future?

Box 38. Guiding questions for defining the context

  • What is the operating environment around the project or programme?
  • How might factors such as history, geography, politics, social and economic conditions, secular trends and efforts of related or competing organizations affect implementation of the initiative strategy, its outputs or outcomes?
  • How might the context within which the evaluation is being conducted (for example, cultural, language, institutional setting, community perceptions, etc.) affect the evaluation?
  • How does the project or programme collaborate and coordinate with other initiatives and those of other organizations? 
  • How is the programme funded? Is the funding adequate? Does the project or programme have finances secured for the future?
What is the surrounding policy and political environment in which the project or programme operates? How might current and emerging policy alternatives influence initiative outputs and outcomes?