I. OVERVIEW, BACKGROUND, MANDATE AND WORK PROGRAMME

OVERVIEW

Introduction

The defining characteristics of the current evaluation are its complexity and breadth. Undertaken with the future in mind, the evaluation has focused on lessons to be learned and recommendations intended to assist in the shaping of programming and in informing planning for the next programming period and beyond.

The evaluation took place as the United Nations Development Programme (UNDP) was midway through a difficult transition from its former status as an agency designed primarily to fund activities carried out by others to one dedicated to playing a more direct, hands-on role. UNDP's priority themes have shifted considerably as a result of decisions emanating from major United Nations conferences over the last few years and subsequent UNDP Executive Board decisions of 1994 and 1995, establishing sustainable human development (SHD) as the overriding mission and poverty elimination as the overall priority of the organization.*

* Executive Board decision 94/12 of 10 June 1994 and 95/22 of 16 June 1995 on the future of UNDP.

It may be thought that some of the commentary and analysis found in the report is a result of applying today's standards and priorities to yesterday's decisions. While the roles of managers and programme officers at headquarters and in the country offices are changing rapidly, as yet, however, the overall structure and internal alignment of operational components as well as support divisions do not correspond to the requirements of the revised mandate.

Some problems and deficiencies in project preparation, design and implementation may be seen as influenced in part by this state of affairs. Certainly, one of the major findings of the evaluation, namely, the weakness in horizontal and vertical coordination and information-sharing within UNDP, is partly attributable to the difficulties of an organization in flux.

Every effort has been made to understand the context in which programmes have been designed and implemented. In any case, the principal concern is not to find fault but rather to draw attention to elements of practice and approach that will require attention as UNDP undertakes the necessary restructuring of its programming work.

Given the responsive character of UNDP's country programmes, it is recognized that intercountry programmes must play an important role in expressing and carrying forward the core values and commitments of UNDP as a development institution. Although the types of projects examined here as well as their substantive focus, size, ambition and range of geographic coverage vary enormously, it is this particular mission, which they are expected to support, that provides common ground and a basic framework within which to carry out the analysis presented below.

The report draws on the collective efforts of the evaluation team, whose names are noted on the title page. Each of the consultants responsible for field research in a particular geographic region prepared a summary report, delivered to UNDP during August l996. These regional reports, which also included documentation and analysis on interregional and global projects investigated in the region in question, provide the source material for much of the content of the present report.

In addition, team members spent ten days together in New York in early August, comparing notes and working on findings that held true across projects and regions. Notes summarizing these discussions have also been a major resource in the focusing of topics addressed here, in the preparation of the case studies highlighted, and in the development of the analysis presented. Finally, it should be noted that the advice and support of Ms. Christine Roth, the project manager, as well as Mr. Carlos Lopes, formerly Deputy Director of the Office of Evaluation and Strategic Planning (OESP), were of special value in assisting with the planning and preparation of this report. This is, in summary, the result of a collective effort.

Organization of the Report

The remainder of chapter one provides the background to the report, including a description of the three different kinds of projects examined, i.e., global, interregional and regional, and a review of the evaluation methodology and the work plan that was developed for putting it into practice.

The principal findings of the evaluation are presented by topic in chapter two. Initially, attention is devoted to the project process, in particular, to project identification, design and preparation as well as to the role of consultations, participation and the issue of ownership by project partners. The review moves on to an assessment of key issues in project implementation before turning to the question of linkages and information-sharing, which are particularly critical in the case of intercountry programmes. The section on substantive programme issues focuses on topics of particular relevance to the mandate of UNDP and dimensions that cut across types of projects and sectors and/or themes for investment. The chapter concludes with a discussion of results and impact, followed by a consideration of issues in organizational culture and structure.

Lessons learned are addressed in chapters three and four. In chapter three, the emphasis is on lessons learned at the strategic or programming level and includes an assessment of the rationale for maintaining intercountry programmes. This is followed by lessons concerning particular approaches that UNDP can apply to optimize its contributions through this type of programming. The initial review is organized by programme type (global, interregional, and regional). The discussion then turns to an effort to capture what is most distinctive about intercountry programmes. In particular, attention is given to the UNDP comparative advantage and the challenges of programming for SHD. Chapter four focuses on lessons learned at the project level, drawn from the discussion of more specific topics and programming issues, following on from the set of issues discussed in chapter two. The recommendations are summarized in chapter five.

Annex I consists of the list of projects in the evaluation sample. The terms of reference (TOR) are provided in annex II. The list of people interviewed at UNDP headquarters is found in annex III while annex IV contains the references for the main documents consulted.

BACKGROUND

The terms of reference (TOR) for the evaluation provide information on the three categories of programming that form the focus for the study (see annex II for the full text of the TOR).

Global, Interregional and Regional Programmes: Purpose, Description and Parameters

Global, interregional and regional programmes are consensually defined as intercountry programmes of UNDP designed to assist groups of countries on a global, interregional, regional or subregional basis. This categorization enables UNDP to provide assistance at the request of at least three governments. The approval of requests and subsequent implementation of programmes take into account equitable distribution of resources among regions.

For the purpose of UNDP programming, the terms global, interregional, regional and subregional have been defined as follows:

Global: Refers to development issues that concern a wide spectrum of countries.

Interregional: Applies to a grouping of countries from two or more regions.

Regional: A region is the area serviced by one of the regional bureaux (presently there are five regional bureaux, namely, the Regional Bureau for Africa (RBA), Regional Bureau for Asia and the Pacific (RBAP), Regional Bureau for Arab States (RBAS), Regional Bureau for Latin America and the Caribbean (RBLAC) and the Regional Bureau for Europe and the Commonwealth of Independent States (RBEC)).

Subregional: A subregion is usually an area of a group of countries within a region, comprising a political, economic or ecological grouping. Subregional programmes do not have a management separate from that of the regional programmes.

Global Programmes

The primary objective of the global programmes is to support and facilitate access by the developing countries to current research on science and technology. At the same time, these programmes seek to enhance the capacities of developing countries to apply research outputs to their most critical development problems. The global programmes of the fifth programming cycle (1992-1996) encompass UNDP support to strategic research activities and the strengthening of the capacity of institutions and individuals in programme countries to undertake strategic research. A total of $94 million was allocated to those programmes for that cycle with approximately two thirds going to research in agriculture and one third for basic health.

Interregional Programmes

Through a pragmatic approach, the interregional programmes have channelled advisory and other services to developing countries in key sectors in all regions. In general, these programmes are often more operational than the global ones and are frequently linked to specific countries, UNDP regional bureaux, other bilateral and multilateral donors, and regional projects. The size of the indicative planning figure (IPF)-funded interregional programmes for the fifth programming cycle was $3l million.

Efforts of both the global and interregional programmes were intended to be as compatible with UNDP regional and country activities as possible.

The guiding principles for the formulation and design of the global and interregional programmes for the UNDP fifth programming cycle have been:

On the basis of these guiding principles, it was proposed that the programmes concentrate their efforts on social development and poverty eradication (to include education, health, HIV/AIDS, water and sanitation, food security, productive employment generation); environment and natural resources; and public-sector management (to include improved economic management, promotion of international trade and investment, improved public administration). In all these cases, gender analysis, capacity-building concerns, participatory approaches, and the promotion of the private sector provided cross-cutting themes receiving special attention. These areas of concentration reflect key aspects of human development such as food security, basic services in health, education and environmental sanitation and income-generation.

Regional Programmes

The pervasive theme of fifth programming cycle regional programmes, as specified in Governing Council decision 90/34 adopted on 23 June 1990, is the promotion of human development through national capacity-building and strengthening in the following areas:

The main focus of UNDP regional programmes has been to strengthen the capacity of national institutions dealing with regional issues and subregional and regional intergovernmental organizations. The following criteria were used to determine whether a project was to be considered regional. Does the project:

Responsibility for Overall Programme Supervision

Regional, interregional and global programmes and projects have been implemented or executed through regional bureaux and specialized or thematic units.

The sources of funding have been IPFs allocated during the fourth and fifth programming cycles and Special Programme Resources (SPR). SPR allocations to support regional and interregional programmes during the fourth and fifth programming cycles were given to BPPS and some headquarters units. However, SPR were not allocated exclusively to regional and interregional programmes; some have been used solely for country activities and in other cases, umbrella interregional programmes have been approved that have provided a framework for country sub-allocations.

A number of the intercountry programmes are funded exclusively or primarily by non-core funding, e.g., the Global Environment Facility (GEF), Capacity 21, Montreal Protocol, Africa 2000, Urban Management Programme.

METHODOLOGY AND WORK PLAN

Preparatory Work

The project began in late February-early March when team leader, Phillip Rawkins, and associate team leader, Fantu Cheru, began work with the project manager, Christine Roth, at OESP in New York. Valuable research support work was undertaken by Asoka Kasturiarachchi of UNDP Colombo, who was on a short assignment to New York.

Initial work focused on: (a) clarifying and focusing an extensive and exhaustive set of TOR; (b) obtaining and reviewing UNDP policy, programme and budgetary documents as well as

Executive Board directives and proceedings; and (c) holding preliminary meetings with those responsible for overall management of the global, interregional and regional programmes.

In the latter part of March, the recruitment of other team members was completed. Eric De Silva, Mary Lisbeth Gonzalez, and Mona Kesseba began their assignments and arrived in New York to participate in the second part of the preparatory process.

The team then began to work together on the following:

Evaluation Strategy

Faced with a massive, unwieldy collection of projects on which to base the research and a highly compressed time frame in which to complete the work programme, the team made a decision, endorsed by OESP, to emphasize depth rather than breadth. Accordingly, the project strategy was to focus on an intensive examination of a small set of projects, selected with a view to enabling the research team to deal with the questions posed in the TOR.

The global and interregional projects were selected. Following research by the team members responsible for each region, a set of regional projects was added to the list. Finally, after extensive discussions, the overall project sample was drawn up (see annex I).

Selection of the project sample reflected a concern for ensuring a reasonable balance among project types, themes, size and scope as well as stage of implementation. A few projects were included largely on the basis of their pioneering and risk-taking qualities. In the selection of projects and themes, the team also took into account the fact that the present evaluation is intended to be forward-looking in its orientation.

Owing to limited time and human resources, no systematic attempt was made to review projects other than those included in the sample. Attention, however, has been drawn to a small number of additional initiatives, chosen to highlight innovative and imaginative approaches adopted despite financial constraints.

Field Research and Interviews

The field research programme was designed primarily to capture the process of the projects from identification to implementation. The goal also was to examine the dynamics of the relationships between the projects and the structures, policies and operational frameworks of UNDP. On this basis, the evaluation team examined some common elements of projects covering a wide variety of sectors and multi-sectoral themes. Since projects are complex, active forms of collaborative work and not simply sets of objectives, plans, activities and budgets, the methodology was designed to capture the "story" of every project examined and enable an analysis based on an assessment of what has happened, why, how and with which results.

Field research was to be conducted in all regions; in many cases, initial interviews also took place at headquarters in New York. Countries to be visited were selected bearing in mind the need for a first-hand assessment of global, interregional and regional projects.

The field assignments for individual team members were finalized with responsibilities allocated as follows: Fantu Cheru covered Africa; Mona Kesseba handled two regions: the Arab States and Europe and the Commonwealth of Independent States; Mary Lisbeth Gonzalez was responsible for Latin America and the Caribbean and Eric De Silva for Asia and the Pacific.

The support of the regional bureaux and other headquarter units as well as UNDP national offices was sought in identifying contacts and setting up interviews. This worked exceptionally well in the case of some projects and some countries and rather less well in others. Guidelines for researchers indicated the desirability of taking a comprehensive approach in identifying and meeting key stakeholders and those responsible for project formulation and implementation. This required an effort on the part of team members to compile a respondent list and complete interviews with all available stakeholders who were capable of providing information on key elements of the project in question. In a few cases, coverage was increased by the use of focus group sessions with small groups of respondents.

In many cases, the limited time available in each country obliged researchers to restrict their interviewing to respondents available in the vicinity of the capital. However, in several cases, it also proved possible to visit project sites and to hold meetings with beneficiaries. On occasion, members of the team used telephone interviews to extend the range of interview sources available.

Wherever feasible, preliminary interviews were conducted with responsible officials in New York. A few additional follow-up interviews were also undertaken by the team leader and associate team leader at headquarters in early July and again in early August to fill in gaps and check preliminary findings.

In view of the comprehensive nature of the TOR, it was inevitable that the questionnaire would be a rather lengthy instrument. In practice, this meant that the team members were obliged to focus on specific areas of the questionnaire or specific groups of questions in dealing with interviewees where the time available was limited. In any case, not all respondents were in a position to answer all of the questions.

The substantive research strategy of this evaluation can be understood readily from a review of the questionnaire, the sections of which deal sequentially with the following sets of issues (each section includes a number of individual questions):

Project Outline

Project Context, Development and Planning

Project Implementation

Overall Assessment