Annex 3. Evaluation terms of reference template and quality standards
The Terms of Reference (ToR) template is intended to help UNDP programme units create ToRs based on quality standards for evaluations consistent with the concepts and terms presented in this Handbook and the UNEG ‘Standards for Evaluation in the UN System’.62
The ToR should also explicitly state a requirement for the evaluation to assess the extent of UNDP commitment to the human development approach and how effectively equality and gender mainstreaming have been incorporated in the design and execution of the project or programme to be evaluated.
In terms of evaluation methodology, the ToR should retain enough flexibility for the evaluation team to determine the best methods and tools for collecting and analysing data. For example, the ToR might suggest using questionnaires, field visits and interviews, but the evaluation team should be able to revise the approach in consultation with key stakeholders, particularly the intended users and those affected by evaluation results (See Chapter 7 for more information on design issues).
The ToR should, at a minimum, cover the elements described below:
1. Background and context
The background section makes clear what is being evaluated and identifies the critical social, economic, political, geographic and demographic factors within which it operates that have a direct bearing on the evaluation. This description should be focused and concise (a maximum of 1 page) highlighting only those issues most pertinent to the evaluation. The key background and context descriptors that should be included are listed below:
More detailed background and context information (e.g., initial funding proposal, strategic plans, logic framework or theory of change, monitoring plans and indicators) should be included or referenced in annexes via links to the Internet or other means of communication.
2. Evaluation purpose
The purpose section of the ToR explains clearly why the evaluation is being conducted, who will use or act on the evaluation results, and how they will use or act on the results. The purpose should include some background and justification for why the evaluation is needed at this time and how the evaluation fits within the programme unit’s evaluation plan (see Chapter 3). A clear statement of purpose provides the foundation for a well designed evaluation.
3. Evaluation scope and objectives
This section defines the parameters and focus of the evaluation. The section answers the following questions:
Issues relate directly to the questions the evaluation must answer so that users will have the information they need for pending decisions or action. An issue may concern the relevance, efficiency, effectiveness, or sustainability of the intervention. In addition, UNDP evaluations must address how the intervention sought to strengthen the application of the rights-based approach and mainstream gender in development efforts.
4. Evaluation questions
Evaluation questions define the information that the evaluation will generate. This section proposes the questions that, when answered, will give intended users of the evaluation the information they seek in order to make decisions, take action or add to knowledge. For example, outcome evaluation questions might include:
Evaluation questions must be agreed upon among users and other stakeholders and accepted or refined in consultation with the evaluation team.
The ToR may suggest an overall approach and method for conducting the evaluation, as well as data sources and tools that will likely yield the most reliable and valid answers to the evaluation questions within the limits of resources. However, final decisions about the specific design and methods for the evaluation should emerge from consultations among the programme unit, the evaluators, and key stakeholders about what is appropriate and feasible to meet the evaluation purpose and objectives and answer the evaluation questions, given limitations of budget, time and extant data.
For example, the ToR might describe in an annex:
6. Evaluation products (deliverables)
This section describes the key evaluation products the evaluation team will be accountable for producing. At the minimum, these products should include:
7. Evaluation team composition and required competencies
This section details the specific skills, competencies and characteristics needed in the evaluator or evaluation team specific to the evaluation and the expected structure and composition of the evaluation team, including roles and responsibilities of team members (see Table 23 on page 142 of the Handbook for more information).
The section also should specify the type of evidence (resumes, work samples, references) that will be expected to support claims of knowledge, skills and experience. The ToR should explicitly demand evaluators’ independence from any organizations that have been involved in designing, executing or advising any aspect of the intervention that is the subject of the evaluation.63
8. Evaluation ethics
The ToR should include an explicit statement that evaluations in UNDP will be conducted in accordance with the principles outlined in the UNEG ‘Ethical Guidelines for Evaluation’64 and should describe critical issues evaluators must address in the design and implementation of the evaluation, including evaluation ethics and procedures to safeguard the rights and confidentiality of information providers, for example: measures to ensure compliance with legal codes governing areas such as provisions to collect and report data, particularly permissions needed to interview or obtain information about children and young people; provisions to store and maintain security of collected information; and protocols to ensure anonymity and confidentiality.
9. Implementation arrangements
This section describes the organization and management structure for the evaluation and defines the roles, key responsibilities and lines of authority of all parties involved in the evaluation process. Implementation arrangements are intended to clarify expectations, eliminate ambiguities, and facilitate an efficient and effective evaluation process.
The section should describe the specific roles and responsibilities of the evaluators, including those of the members of the team, the Task Manager, the management of the commissioning programme unit and key stakeholders. The composition and expected roles and responsibilities of the Advisory Panel members or other quality assurance entities and their working arrangements should also be made explicit. In case of a joint evaluation, the roles and responsibilities of participating agencies should be clarified. Issues to consider include: lines of authority; lines of and processes for approval; and logistical considerations, such as how office space, supplies, equipment, and materials will be provided; and processes and responsibility for approving deliverables.
10. Time-frame for the evaluation process
This section lists and describes all tasks and deliverables for which evaluators or the evaluation team will be responsible and accountable, as well as those involving the commissioning office, indicating for each the due date or time-frame (e.g., work plan, agreements, briefings, draft report, final report), as well as who is responsible for its completion. At a minimum, the time breakdown for the following activities should be included:
In addition, the evaluators may be expected to support UNDP efforts in knowledge sharing and dissemination (see Chapter 8). Required formats for the inception reports, evaluation reports and other deliverables should be included in the annexes of the ToR for the evaluation being commissioned. This section should also state the number of working days to be given to each member of the evaluation team and the period during which they will be engaged in the evaluation process (e.g., 30 working days over a period of three months).
This section should indicate total dollar amount and other resources available for the evaluation (consultant fees, travel, subsistence allowance, etc.) This is not a detailed budget but should provide information sufficient for evaluators to propose an evaluation design that is feasible within the limits of available time and resources. If the available amount is not sufficient to ensure the high quality of evaluation products, discussions can take place between the evaluators and the commissioning unit early on in the process.
12. ToR annexes
Annexes can be used to provide additional detail about evaluation background and requirements to facilitate the work of evaluators. Some examples include:
Table A. Sample evaluation matrix
62. UNEG, ‘Standards for Evaluation in the UN System’, 2005. Available at: http://www.unevaluation.org/unegstandards.