Annex 5. Selecting evaluators: Individuals versus firm

The following are some of the questions to be asked in determining the size and composition of the evaluation team and advantages and disadvantages in hiring individuals65 and firms.

  • How many are in the team and what skill mix is required? The number of evaluators in the team depends on a number of factors. Multi-faceted evaluations need to be undertaken by multi-disciplinary teams. The members selected must bring different types of expertise and experience to the team. The ideal team should represent a balanced mix of knowledge of evaluation methodology required for that particular evaluation, knowledge of the subject to be evaluated, knowledge of the context in which the evaluation is taking place or familiarity with comparable situations, and knowledge of cross-cutting issues in evaluation, such as gender.
  • What is the mix of internal and external perspective? If possible, the evaluation team should include at least one national team member. Ideally, an evaluation team combines national members (who bring the local perspective and experience) and international members (who bring the external perspective). 
  • Should we use a firms or individuals? There are two ways to establish an evaluation team: one is to select individual consultants to form a team, another is to ask a firm to propose a team. Table B gives an overview of advantages and disadvantages for both options, which the committee should take into consideration.
Table B. Advantages and disadvantages of individuals versus firms

 

Individuals

Firms

Advantages

  • Individuals may bring specialized expertise and many years of experience in particular subjects.
  • The variety of backgrounds of individual team members contributes to debate and discussion that can enrich the exercise.
  • May be less expensive.
  • May be more amenable to last-minute changes in the ToR or other arrangements.
  • Especially for nationals, the evaluation process may provide opportunity for capacity development and learning amongst individual experts.

 

  • Fees are agreed as a package that is unlikely to vary, unless there is a change in the ToR.
  • Members of the team are used to working together.
  • The firm assures the quality of the products.
  • A multidisciplinary approach is guaranteed.
  • Hiring procedures, although they can be longer than for an individual, are usually easier.
  • The firm develops the methodology or proposal for the evaluation.
  • In the event of sudden unavailability (e.g., illness) of an evaluator, the firm is responsible for providing a substitute.

Disadvantages

  • Identification of individual consultants is time-consuming and there are risks in selecting evaluation team members solely on the basis of claims made in their applications. 
  • A team of professionals that have never worked together can have difficulty developing a sense of cohesiveness and coherence in their work, and internal conflicts can affect progress.
  • Changes in the schedule can result in additional costs in fees, per diem and travel arrangements.
  • Logistics to be provided by the country office.
  • The fees may be higher, as the firm's overhead will be included.
  • If the firm has been overexposed to the topic or the organization, the credibility of the exercise can be compromised.
  • Team members tend to have similar approaches and perspectives, thereby losing some of the richness of different positions. 
  • Bidding procedures can be lengthy and cumbersome.
  • Firms may have difficulty supplying a mixture of nationals and internationals.