4.4 Use of monitoring data for management action and decision-making

Data and information on progress towards results are gathered, reviewed and used at the project, outcome, sectoral and programme levels. These entities are interconnected and reinforce each other. There is a two-way flow of information among them with the following common objectives:

  • Clarifying and analysing progress, issues, challenges and lessons
  • Precipitating actions and decisions including effecting changes in plans and resources as required
Project level

Monitoring data normally aggregates from project level to higher level results. At the project level, the use of monitoring information can be summarized as follows.

The first monitoring action at the project level is to be clear of what is expected in terms of project-specific results and what is to be done with respect to monitoring actions. At the beginning, projects should: have a

clear scope (that continues to be clear throughout the project); expected deliverables and how these contribute to the higher level results; ensure that cumulative annual targets are adequate to produce the envisaged outputs; and ensure that they lead to the delivery of planned outputs in the agreed time-frame. This information is initially captured in the project results framework and its M&E framework. This process should be repeated at each annual project review to continuously validate that delivery of outputs is on schedule and remains relevant. If this is not the case, higher level boards or committees should be notified so that any implications on the overall planned results can be reviewed for modifications, new time-frames and costs.

Monitoring data should be collected according to the AWP, and in the case of UNDP, by using Atlas-generated quarterly progress reports. The project should review the data to:

  • Revalidate if the project and programme results logic remain valid in light of the operational experience and evidence.
  • Discern what issues have emerged during implementation: Have the foreseen risks and assumptions materialized? Have other unforeseen challenges, opportunities and risks materialized? Are these being managed?

Progress towards generating outputs and their continued relevance to the outcome and issues should be synthesized and forwarded to the agency to which the project reports and to the respective outcome or sectoral monitoring mechanism. On the basis of that monitoring data, the project management and board or steering committee should re-confirm that the delivery of outputs is on schedule and that the project is contributing towards the desired outcomes. If not, they should determine what changes are needed. If revisions to plans are needed, then the project management should draft the revisions, including the results framework with new cost estimates, annual targets and so forth, to facilitate decision-making at higher levels. Such information could be provided at agreed intervals such as quarterly, semi-annually, annual or on an as needed basis.

Outcome level

Sectoral and outcome-level coordinating mechanisms play a critical role in results monitoring and developing capacities for monitoring. They forge partnerships around initiatives supported by partners to achieve common results, provide oversight in a collective spirit, make linkages to national systems and national development goals, and promote the development of monitoring capacities.

At the beginning of the programme or project implementation, the existence of such outcome-level monitoring and oversight mechanisms should be verified. If such mechanisms do not exist, then arrangements should be made to set up such groups through engagement with national partners. As interim measures, UN Theme Groups could be set up in accordance with UNDG (CCA and UNDAF) Guidelines. The outcome and sectoral monitoring mechanisms should take the following actions:

  • Ensure that all those who are contributing to the outcome are included in the group. For UNDP, this should answer the question whether UNDP is engaged with the right partners to deliver outputs and to achieve outcomes.
  • Agree on regular interactions and a plan of action to ensure that coordination and monitoring mechanisms remain efficient and effective.
  • Review the components of the outcome (outputs and other activities) and ensure that outputs to be produced are sufficient to bring about the outcome and sustain the benefits.
  • Ensure that the results plan for outcome indicators, targets, risks and assumptions are valid, adequate and managed.
  • Promote development of national capacities in monitoring.
  • Agree on a practical arrangement to coordinate the functioning of the outcome group. The outcome group should ideally be led by a national entity. However, UNDP may also offer such services.

The sectoral or outcome coordinating mechanism should continually assess the status of outputs and related initiatives by partners—all of which contribute to an intended outcome. It does so by examining information from all relevant projects, national reports, donor reports and other sources. It should review the findings of quarterly and annual reviews pertaining to the outcomes and identify lessons that are to be fed back into programming, and serve as a vehicle for ensuring documenting and disseminating lessons learned. It also serves as the focal team for outcome evaluations. Specifically it should:

  • Review and assess connected projects and provide feedback to all relevant partners upon receipt of relevant reports, notably the APRs from each contributing project.
  • Consider any changes needed in each constituent project and in overall approach in order to achieve the outcomes, the consequences of the necessary changes, and take appropriate action to ensure achievement of the outcomes.

It is important to keep in mind that the outcome and sector-level coordinating mechanisms are ‘larger’ than the United Nations and UNDP programme, as they focus on the achievement of the national outcomes. Hence, the United Nations and UNDP is one of many contributors towards the achievement of these outcomes. Ideally, the outcome and sector-level coordinating mechanisms should not be a United Nations or UNDP management arrangement but an existing national structure that is charged to coordinate the sector within the national context.

Programme level

Each partner (such as UNDP) that contributes to one or more outcome typically has its own arrangements to plan, implement and monitor the contributions it is making to results. For UNDP at the country level, this is the function of the CPAP and its monitoring and annual review. The M&E framework, which is the CPAP monitoring framework, forms the basis for this purpose.

The primary question to address at the programme level is: Does UNDP, as a partner, ensure that its programme is effectively contributing to the planned UNDP country programme, UNDAF and national results within the agreed partnership arrangements? The same principle applies for UNDP regional programmes, global programme and the Strategic Plan. Furthermore, it should also ascertain whether or not country programmes, regional programmes and the global programme are contributing to the objectives and envisaged outcomes of the Strategic Plan.

The following steps are necessary to organize programme-level monitoring:

  • Obtain monitoring information for each UNDP funded project through the respective quarterly progress report (if used), APRs and other related activities, such as soft assistance relevant for a given outcome. This should answer the questions: What progress has UNDP made in delivering the agreed CPAP outputs in the reference period? What progress has UNDP as a whole made towards achieving the CPAP outcomes? What are the programme-level issues that require action? What are we learning as a programme?
  • Determine if outputs being generated with UNDP support remain valid and contribute to achieving corresponding outcomes.
  • Participate in dialogue with relevant stakeholders at the outcome, sector and national level.
  • Determine if other partners are contributing as planned and identify gaps to be addressed and opportunities for forging stronger partnerships.
  • Triangulate monitoring information to obtain a more objective assessment of the UNDP contribution to each outcome. Identify issues and changes that are necessary to further dialogue at the respective outcome monitoring mechanism. Analyse to what extent UNDP has integrated key concerns such as capacity development, gender equality, national ownership and South-South cooperation.
  • Summarize key relevant points for the programme as a whole for corporate reporting purposes and decision-making at the annual programme review.
  • A separate annual review meeting on the UNDP country programme may not be necessary if issues pertaining to the UNDP programme and related decisions could be covered at the UNDAF annual review. It might be helpful to hold a one-day UN programme review at the annual review, where a half day focuses on the UNDAF and the other half day focuses on respective agency programmes.
  • Implement the necessary changes agreed at annual reviews.

The same analytical work should feed the corporate learning and reporting processes. For UNDP, this means that the managers of country, regional and global programmes should feed the findings of this analysis into the RBM Platform to report on progress against the Strategic Plan.

Annual review with key stakeholders

The annual review with the participation of all key stakeholders is a key monitoring event at the national programme or UNDAF level. It is the culmination of monitoring activities that started at the project level and cascaded upwards through the outcome and individual partner programme level during the year. The annual review facilitates a dialogue among senior managers to assess progress towards results (outputs and outcomes). It is also a forum that is used for building a stronger mutual understanding and consensus among partners on the issues directly relevant to achieving the planned results and for making key high-level decisions. Annual reviews are ideally held towards the end of the year, and the discussions are meant to guide and approve plans for the following year.

Annual reviews have to be well planned in order to extract the best results from them. The following should be considered in the preparation for the annual review:

  • It is essential that the annual review is conducted based on objective monitoring data and analyses prepared by all connected projects (for example APRs) and finalized after consultations with relevant stakeholders. The following actions could be helpful:
    • Based on the APRs and project board or steering committee and outcome groups’ or boards’ findings and recommendations, each partner organization should present a synthesis of its own key points—including an assessment of its contribution to the outcomes and other issues that need to be discussed at the annual review. These syntheses should be succinct and made available to annual review participants prior to the meeting.
    • Given the time constraints at the annual review meeting, if deemed necessary, organize prior consultations among relevant partners to ensure deliberations at the annual review will be efficient, avoid potential conflicts, and lead to decisions and clear follow-up actions on the subsequent year’s work programme.
  • The annual review should be organized by the relevant national partner and carried out in an inclusive and practical manner. Depending on national capacities, and in consultation and with the leadership of the key national partners, another partner (for example UNDP) may organize or assist in the organization of the annual review. The participants should be at the decision-making level of each participating partner. Its success often depends on how well the partners have been involved in the lead up to the annual review and are informed on the issues to be discussed. Ideally, many of the issues to be addressed at the annual review should already have been discussed, for example, in regular monitoring events such as field visits or in prior discussions on the APRs at the project or outcome level. A focused approach is recommended for the annual review so that the key issues and outcomes are addressed. The following actions may help in this respect:
    • The agenda of the annual review should be prepared carefully, giving priority to those items that require collective review and decision-making by the partners at the annual review meeting. Ensure sufficient time for dialogue and provide background information for each agenda item in advance.
    • Ensure that the annual review process will lead to decisions and agreements on: the current status of the achievement of the results being pursued; any changes to overall results frameworks; and updated AWP for forthcoming year.
Global and regional programmes (of UNDP)

At a minimum, an annual review of the global programme and of each regional programme must be held. These annual reviews are informed by a variety of information sources, including APRs of constituent projects.

Box 25. Typical UNDAF annual review process and lessons learnt
At the country level:
The UNDP annual review process is linked to the UNDAF annual review, which is the once-a-year opportunity for all agencies and national partners to review the UNCT contribution to achievement of national goals based on the UNDAF Results Matrix. In consultation with national partners, the UNCT decides on the meeting’s scope and modalities.

The UNDAF annual review should provide the UNCT and national partners with:

  • A yearly update of overall progress vis-à-vis the UNDAF Results Matrix
  • Validation of conclusions and recommendations that should feed into annual planning processes

UNDP contributes to the annual UNDAF review through the annual results reporting in CPAP. The annual report of CPAP is prepared from: analyses that originate from project APRs and the project boards; coordinated comments on each outcome by sectoral or outcome coordinating mechanisms (including national coordination mechanisms and UN Theme Groups) to reflect progress towards outcome at outcome levels; and project and outcome evaluations or any other relevant outcome and project reviews that have been carried out during the year, including those carried out by other partners.

Substantively, UNDP contribution to UNDAF annual review—a synthesis of the CPAP annual report—includes: a brief assessment of the achievement of annual targets of all UNDP funded activities in the context of achieving UNDAF outcomes;  operational issues of the CPAP implementation; and any modifications to the existing CPAP that would require agreement of non-UNDP partners.

Some general lessons learned from conducting UNDAF annual reviews, based on the experience of the Solomon Islands:

  • Use the government’s National Development Plan as the organizing principle for the review—Organize reporting and deliberations on the basis of the National Development Plan. This may require additional work on the part of UN system and other partners, but the extra effort is highly justified by the resulting increased national ownership.
  • Be strategic—Presentations on individual UN organization programmes would be uninteresting.  Avoid long lists of outputs by individual partners. Such information could be presented as background documents and referred to in the meeting. Focus on the likely development changes in relation to development indicators.
  • Repetition makes things easier—It is difficult for many organizations to cooperate on annual planning and review exercises—each agency has its own formats, timelines and terms. 
  • Standard formats should be devised and discussed at the earliest possible date—While it is tempting to focus more on the structure of the meeting and agenda, it is the finalization of the annual reports and AWPs that make up most of the work of the review exercise. Thus, the earlier this work is started, the better.   
  • Reduce transaction costs—Use video and web-conferencing for consultations among UN partners.
The United Nations is stronger together than separate—Taken as a whole, aggregated support of the UN system in a country could be on par with other major external partners. This enhances the UN system’s position and also underpins the principle of Delivering as One.
 
Follow up to annual review
  • Agendas and records of annual review meetings should be documented, circulated among all partners and agreed upon by them.
  • Revise the AWP subsequent to, and in line with, the decisions of the annual reviews. It should be approved, preferably in writing, by all the partners involved, typically at the project board level. The M&E frameworks at programme (CPAP) and project levels and the accompanying AWP monitoring tool should be prepared thereafter in readiness for monitoring purposes in the subsequent year.
  • For UNDP, when the annual review is completed and new work targets for the subsequent year are agreed upon, the following processes are triggered: updated AWPs for projects are finalized with the new annual targets and signed; results of the review year is updated in the RBM Platform for corporate annual reporting; and newly agreed targets are set using the RBM platform for subsequent annual reviews.
  • Coordinate any changes with the outcome or sector level committee to ensure that all stakeholders are aware of any changes.
Use of monitoring data in evaluations

Effective monitoring generates a solid data base for evaluations. Data, reports, analysis and decisions based on monitoring evidence should be retained with a view to making them easily accessible to evaluations.