EXECUTIVE SUMMARY

Context and Purpose

Principal Findings

• UNDP and Its Current Agenda: the Relevance of Intercountry Programmes
• Characteristics of Successful, Effective Projects
• Barriers and Impediments to Quality of Performance

Major Recommendations

• Global and Interregional Programmes
• Regional Programmes
• General Recommendations for All Intercountry Programmes
• Project Cycle and Substantive Issues
• UNDP and the Pursuit of the SHD Mandate

---

Context and Purpose

The evaluation of global, interregional and regional programmes took place between February and September 1996 when UNDP was midway through the process of determining strategic directions for its programming work. Various initiatives were also being considered for some elements of restructuring of the organization in keeping with its new objectives, concentrating on sustainable human development (SHD) and poverty eradication. The study focuses on the extent to which the programmes examined have enhanced the capacity of the United Nations Development Programme (UNDP) to respond to its objectives. Many of the UNDP fifth programming cycle priorities have been shaped by the recommendations and identification of issues arising from the major United Nations conferences of recent years. Consequently, the relationship between UNDP's priority themes and the objectives, activities and results of the programmes under review has also been evaluated. In addition, project performance and relevance, measured against both organizational priorities and project goals, have been assessed.

The evaluation was planned and developed with a forward orientation, seeking to determine lessons and recommendations that will inform decision-making regarding the overall shape and character of future programmes. At the same time, an effort has been made to draw practical, specific lessons to assist in the preparation, design and implementation of individual projects.

Principal Findings

UNDP and Its Current Agenda: the Relevance of Intercountry Programmes

On the basis of the findings of the study, the case has been made that there is a particular place for intercountry programmes within the broader UNDP repertoire. By contrast with country programmes, intercountry programmes cannot, for the most part, be directly responsive to the specific concerns of individual governments in a traditional way. In addition, regional programmes in particular cannot be viewed as a pool of money to be accessed by United Nations specialized agencies or the regional commissions with their own programming priorities. Instead, intercountry programmes should endeavour to address issues of broader global or regional significance while simultaneously seeking to carry forward the SHD agenda.

UNDP's basic objectives are confirmed by the Executive Committee and endorsed by the broader membership of the Executive Board. This provides a broad-based consensus on the overall thematic directions to be pursued by UNDP. Intercountry programmes offer the most direct opportunity for UNDP to demonstrate how its SHD and poverty eradication agendas can be pursued. They also provide a context in which to introduce and assess new approaches and methods in grappling with critical development issues.

There is a place for projects, such as those supported by the global programme, which emphasize the strengthening and use of research capacity in the South (as well as in Europe and the Commonwealth of Independent States), but projects of this kind should emphasize applications, dissemination and linkages with beneficiaries and end-users.

Projects need not be managed by UNDP. Indeed, many of the most effective and imaginative projects examined here are handled at arm's length from UNDP by other organizations. Nevertheless, regardless of the extent to which it is directly involved with execution and implementation, UNDP must ensure that projects are planned with a view to learning. It must also have a hand in the shaping of broad project objectives and should ensure that they, as well as expected results, are well linked to broader organizational objectives and values.

Intercountry programmes are best viewed as demonstration projects that seek to advance practical approaches to addressing critical SHD/poverty eradication problems. To meet this objective, projects should incorporate mechanisms for the capturing and sharing of experience and the dissemination of lessons learned. UNDP itself must use its intercountry programmes as a platform for learning, but it must also ensure that project partners as well as other donors and stakeholders can also share the benefits of what may be learned.

The emphasis on learning and change is the core component of the very rationale for intercountry programmes. It cannot be merely an add-on. Whether or not projects can afford to be built in this way is not an issue. Given that projects are investments dealing with small elements of much larger problems, however worthy the individual project, it represents an investment that may very well be wasted without the incorporation of a learning strategy as a core element of design.

It is apparent from the findings of the evaluation that UNDP cannot afford the luxury of a scattered approach to intercountry programmes. While there is room for variation and while priority should be given to new thinking, the range of themes/sectors addressed must be reduced drastically. Fewer, better-prepared projects will serve the interests of the organization and its partners better. Funds should be set aside in any given year to support smaller pilot initiatives, which might prove themselves worthy of consideration for more substantial support a year or two later.

The evaluation identified some outstanding examples of projects, large and small, that represented noteworthy practical models of conceptualization and action in addressing key problems central to the SHD agenda. For all this, on a broader front, many projects failed to deal directly and head-on with SHD issues. This was largely a result of insufficient attention to pre-design work and to properly focusing and designing projects. In addition, it was found that, in many cases, low priority had been given to building the partnerships required at the national and local levels to ensure that projects were closely linked to the SHD needs of the countries involved.

In many cases, it is apparent that UNDP needs to build cooperative arrangements with other organizations with strong networks at the country, regional and global levels and with the substantive capacities in SHD and poverty eradication that UNDP has been found to lack.

The evaluation results indicate that UNDP staff, both at headquarters and in the field, require better support and better training and preparation if they are to assist the organization properly in fulfilling its mandate. Problems in management, ongoing lack of commitment to consultation and partnership, inadequate monitoring arrangements, under-budgeting, and lack of attention to assessing performance were found to undermine seriously programme quality and relevance.

Characteristics of Successful, Effective Projects

In the evaluation, examples of best practices have been found in all categories of programming: global, interregional and regional. In summing up these best practices, it is possible to provide a list of characteristics that may be seen as associated with successful, effective projects. These include:

• careful, thorough preparation, including broad-based consultations and a careful assessment of the capacity of selected partner organizations;
• building on previous activities and lessons learned;
• involvement of stakeholders and/or beneficiaries in the implementation of project design;
• a clear definition of the problem to be addressed and a focus on problem-solving at the heart of project design and reflected in project objectives;
• realistic setting-out of expected results (and, ideally, performance indicators);
• a coherent, practical implementation strategy linked to objectives;
• a budget based on a careful, realistic assessment of probable costs and structured and balanced in keeping with priorities and the relative needs of the activity sets and primary objectives;
• a strategy for anchoring the project at the country level and an approach that emphasizes involvement of country offices and resident representatives from an early stage;
• a clear definition of roles and responsibilities regarding overall project management, accountability, implementation and achievement of results;
• a straightforward structure for project governance;
• a realistic allocation of resources (human and material) to support management, monitoring and backstopping;
• an effective strategy for information-sharing and learning, built on the experience of other projects;
• a realistic strategy for capacity development where capacity development is relevant to the project and where the project has resources to support it;
• a project document that sets out all elements of design clearly and coherently and that is designed as a management and monitoring tool.

Barriers and Impediments to Quality of Performance

Just as the evaluation has been able to point to a series of best practices, so has it also identified a pattern of factors that consistently interfere with the pursuit of quality in programming. The principal factors are:

• too many projects and a scattering of projects by topic and theme;
• under-budgeting of projects;
• insufficient time and resources given to preparation, design and the appraisal process;
• inadequate attention to adopting a consultative and participatory approach in working with partners and stakeholders;
• inadequate internal staff allocations to supervise project appraisal, design, management and monitoring;
• poor definition of stakeholders and beneficiaries;
• insufficiently focused projects;
• absence of common standards for project quality and of a mechanism for assuring it;
• the lack of an organizational commitment to quality through the work of project assessment committees;
• lack of fit between organizational processes and the needs of project planning and implementation;
• absence of a commitment to learning and of a structure within and across projects to facilitate information collection, exchange and learning;
• weak management capacities within UNDP, both in New York and in country offices, in supporting intercountry programmes;
• limited and poorly used professional and technical capacities.

Major Recommendations

Global and Interregional Programmes

1. UNDP should continue to support global/interregional programming as a central component of its overall portfolio.

To be effective, global programmes (including the range of fifth programming cycle activities covered by global/interregional) should observe the following guidelines:

• they should be designed as demonstration projects, seeking to build in learning approaches and mechanisms for the sharing of experience;
• they should be multi-disciplinary wherever possible;
• they should be regarded as a context for innovation and the development/testing of new approaches;
• they should be undertaken in full consultation with project partners and stakeholders at both the design and implementation stages;
• at the planning stage, every effort should be made to consult with and involve regional bureaux;
• global projects should be designed with a view to seeking opportunities for linkages with projects at the regional and country levels;
• project planners should work in partnership with resident representatives and country offices in stakeholder identification, capacity assessment, the planning of consultations and the design and implementation of country-based components of projects.

2. A strategic framework should be established for global programmes, concentrating on a limited range of priorities and themes to be addressed in each programming period.

CHARACTERISTICS OF SUCCESSFUL, EFFECTIVE PROJECTS	BARRIERS AND IMPEDIMENTS TO QUALITY PERFORMANCE
Careful, thorough preparation, including broad-based consultations and careful assessment of the capacity of selected partner organizations;	Insufficient time and resources given to preparation, design and the appraisal process, resulting in poorly focused projects;
Building on previous activities and lessons learned;	Too many projects, a scattering of projects by topic and theme and under-budgeting of projects;
Involvement of stakeholders and/or beneficiaries in project design and implementation;	Absence of common standards for project quality and of a mechanism for assuring it;
A specific definition of the problem to be addressed and a focus on problem-solving at the heart of project design and reflected in project objectives;	Poor definition of stakeholders and beneficiaries;
Realistic setting-out of expected results (and, ideally, performance indicators);	Inadequate attention to adopting a consultative and participatory approach in working with partner(s) and stakeholders;
A coherent, practical implementation strategy linked to objectives and a budget based on a careful, realistic assessment of probable costs and structured and balanced in keeping with priorities and the relative needs of the activity sets and primary objectives;	Insufficient internal staff allocation and poor use of professional and technical capacities to supervise project appraisal, design, management and monitoring;
A strategy for anchoring the project at the country level and an approach that emphasizes involvement of country offices and resident representatives from an early stage;	Inefficiency in integrating organizational processes and the needs of project planning and implementation;
A clear definition of roles and responsibilities regarding overall project management, accountability, implementation and achievement of results;	Weak management capacities within UNDP, both in New York and in country offices, in supporting intercountry programmes;
A straightforward structure for project governance;	Absence of a commitment to learning and of a structure, within and across projects, to facilitate information collection and data exchange.
Realistic allocation of resources (human and material) to support management, monitoring and backstopping;
An effective strategy for information-sharing and learning, built on the experience of other projects;
A realistic strategy for capacity development where capacity development is relevant to the project and where the project has resources to support it;
A project document that lays out all the elements of design clearly and coherently and that is designed as a management and monitoring tool.

3. For global programmes, a small fund should be set aside to support the exploratory stage of innovative initiatives that show promise but that have yet to prove themselves. Such initiatives should include strong monitoring components.

Regional Programmes

1. Regional bureaux should undertake programme planning within an overall strategic framework that will apply to all regions.

While there should be variations in the character and form of programming from one region to another, guidelines should be developed, for approval by the Executive Committee, that will facilitate the development of regional programmes concentrating on a narrower range of priorities and themes.

2. Regional programmes, like global programmes, should use a demonstration and learning approach in support of the achievement of specific, SHD-related objectives. Regional programmes lack the resources to develop projects with a service-oriented approach.

General Recommendations for All Intercountry Programmes

1. The primary mandate of all intercountry programmes should be to investigate and demonstrate the utility of alternative methods of addressing core SHD/poverty eradication priorities. To put this mandate into practice, programme planners require far better support than that currently available in operationalizing SHD concepts and principles.

2. In planning intercountry programmes, UNDP should give particular attention to efforts to facilitate joint learning approaches with other donors, country partners and other stakeholder organizations in working in particular thematic areas.

3. UNDP should undertake a realistic appraisal of its in-house capacities to provide professional support and direction to SHD programming. Such an assessment of core capabilities would facilitate a better understanding of those areas of activity where UNDP would be best advised to form alliances with other organizations at the global and regional levels in the building and delivery of effective programmes.

4. Whatever its role in cooperative funding, management and execution of projects, UNDP should always ensure a sufficient internal allocation of human resources and money to permit its staff to play a proactive role in overall management and monitoring.

Project Cycle and Substantive Issues

1. The administration should take steps immediately to produce improved guidelines for the preparation of projects and project documents in consultation with staff in regional bureaux and the Bureau for Policy and Programme Support (BPPS).

In addition to the core components always included in accordance with existing guidelines, it should be stipulated that the following are essential elements of a sound project document:

• a clear statement of the broader development problem to be addressed;
• a clearly focused specification of a more limited development objective or goal;
• properly articulated objectives and activities, which follow logically from the development objective, along with expected results;
• an implementation strategy, accompanied by a detailed work plan, to be updated annually;
• a staffing plan, associated with the work plan;
• a management plan, with clearly stated roles and responsibilities for all project partners and principals;
• a straightforward description of the governance structure for the project.

2. Project documents should include a systematic strategy for monitoring and evaluation with separate budget lines for monitoring and for evaluation.

3. New procedures and monitoring mechanisms should be developed to make certain that project approval committees operate effectively in ensuring quality control and the achievement of high and comparable standards across the portfolio of UNDP intercountry projects.

4. The administration should take steps to consult with the regional bureaux and BPPS to put into place guidelines that, in combination with strategic frameworks for programmes, will reduce the number of intercountry projects and also help to ensure an adequate budget for each project approved.

5. UNDP should establish a consultative/advisory body with representation from the regional bureaux and BPPS to review and monitor developments in the field of intercountry programmes. The advisory committee would be briefed on newly funded initiatives and the results of project and thematic reviews and evaluations. It would also undertake/ commission, through and in consultation with OESP, studies of various kinds regarding overall trends and performance in intercountry programmes. In addition, it would take on responsibility for information-sharing on intercountry programming throughout UNDP and consider issues of common concern.

6. Consultations and Participation

• Project preparation guidelines should emphasize the importance of using a participatory and consultative approach at both the design and implementation stages. Project approval committees should ensure that proper steps have been taken in this regard.
• The project preparation and project document guidelines should address the importance of careful, detailed work in specifying beneficiaries and clearly identifying stakeholders. This will also result in better-focused projects.

7. Management and Implementation

• All projects should be expected to provide a clear, unambiguous statement concerning roles and responsibilities for project management and decision-making.
• UNDP should also address the issue of more clearly distinguishing between the responsibilities associated with execution and those associated with implementation (the distinction between the two is made on a daily basis, yet in practice, the boundary is unclear).
• UNDP should take appropriate steps to ensure that adequate human and financial resources are allocated to facilitate the undertaking of essential monitoring and supervisory responsibilities by UNDP headquarters staff.
• UNDP should note in the strategic frameworks and guidelines for intercountry programmes the importance of continuing consultation with involvement of resident representatives.
• Advice should be provided to make certain that project design and budgets make provision for supporting the role of country offices in intercountry programmes.
• UNDP should ensure that appropriate training and preparation are provided to national officers engaged in supervision/monitoring of intercountry programmes.

8. Monitoring, Evaluation and Performance Assessment

• Programming guidelines should be developed (and enforced) to ensure that all intercountry programmes include a satisfactory strategy for monitoring and evaluation.Wherever possible, project partners and stakeholders should be involved ongoing monitoring and in planning for project evaluation.
• In association with the monitoring and evaluation strategy, projects should develop, at the design stage, a set of milestones and performance indicators as well as a statement of expected results to provide a foundation for monitoring and evaluation. Project preparations and consultations should seek to build agreement and understanding among project partners on the relevance of the set of indicators selected and their utility in supporting mutual learning.

9. Capacity Development

• The administration should consult closely with BPPS (and particularly the Management Development and Governance Division (MDGD)) to correct the weakness in project design and performance (in the intercountry sphere, at least) with regard to capacity development.
• OESP and MDGD should work together in an effort to capture lessons from best practices and use the knowledge gained thereby in training and in the development of manuals and other programming support tools.

10. Gender

It is strongly recommended that attention be given by the administration and management at the bureau/division level to the absence of incentives for dealing adequately with the gender dimension in intercountry programmes. At the same time, attention should be given to the lack of preparedness of staff to cope with gender issues and gender mainstreaming.

UNDP and the Pursuit of the SHD Mandate

1. Learning

• As a matter of urgency, UNDP should develop guidelines for project planners on building intercountry projects around learning approaches. Such guidelines should be as practical as possible.
• The administration should give serious attention to the need to develop incentives and support systems to facilitate and encourage a positive attitude by staff in recognizing learning from project experience, as well as information-sharing across projects and components of UNDP, as an essential building block of an organization committed to SHD.

2. SHD Leadership: Focus and Concentrate

• In its provision of guidance to intercountry programmes through strategic framework documents and guidelines, UNDP should take into account the fact that, if it is to be a leader in SHD, it must concentrate and focus its intercountry programming resources.
• UNDP should take a pragmatic, entrepreneurial role in defining a niche and a development gap. This will involve identifying and securing partners and then devising strategies to fill a gap not adequately filled by any other donors. This can only be possible with overall guidance and supervision from headquarters along with focused professional support and constructive dialogue with programme planners and international partners.
• The administration should give attention, as a matter of urgency, to improving quality control, management, monitoring, information-sharing and organizational learning.