There is a considerable difference between conducting and utilizing qualitative evaluations and quantitative evaluations. In practice a good evaluation often will combine the two types of evaluation, unless there are clear limitations or restrictions due to skills of personnel involved or a cost-or time-factor to be observed.

Qualitative Evaluations

Qualitative evaluations (e.g. working with case-studies) tend to give insight in possible causal relationships that may exist. In other words, such studies suggest answers on the question of ?why?. Such answers are of particular relevance for evaluations designed primarily for learning purposes. Qualitative evaluations require evaluators that are highly skilled in ?distant empathy?. The empathy is necessary in order to grasp dynamics that caused things to go as they went. The ?distant? part is required in order to place these dynamics in their context, to recognise alternatives that were not taken. Not at least, to discover motivations and (hidden) agenda?s. Many evaluators and principals prefer qualitative evaluations. However, from a methodological point of view, the qualities, time and funds required are often (heavily) underestimated.

There is one more major drawback for qualitative evaluations. That is the ability to generalize the case-study findings. How to ?prove? that the conclusions reached on basis of selected observations and/or responses also are valid for other parts of the programme, for other groups, organisations, individuals? In order to make such wide bearing plausible, a number of tools may be used but really hard positive conclusions on generalisability will be difficult to formulate. In spite of these draw-backs it should be underlined that there is hardly an alternative: if we wish insight in factors and causalities at work, we need qualitative evaluation.

Quantitative Evaluations

Quantitative evaluations are best when accountability and generalisability are the primary issues. This is also the case because for accountability we need to generalise and qualitative studies do not usually provide data that are easy to generalise. The number of beneficiaries used, the type of target group reached, the geographical distribution, etc.

Quantitative evaluations are typically associated with the questionnaire. Where in qualitative evaluations the professional (empathical) skills of the individual evaluator(s) can be considered as a cornerstone, in quantitative evaluations the formal methodological skills as well as the organisational capacity of the evaluating organisation is of crucial importance: the construction of the questionnaire, the training of those involved, the qualities of a help-desk, the processing of data, etc.

Quantitative evaluators sometimes give the impression (or have the feeling) that they are more ?scientific? apparently because they are dealing with numbers, statistical ?significance?, objective comparisons, etc. In theory, such claims may be valid when on basis of perfect sampling, generalised conclusions can be reached. However, apart from various other problems that may occur, sampling is rarely perfect, if only because of financial and practical limitations.

In general: quantitative evaluations tend to have great problems in illustrating the dynamics at work, the role of parties, organisations and persons and their motivations. They are much more suited for more objective descriptions. At best they give probabilities rather than causalities.

The best mix

Mixing qualitative and a quantitative approaches does not occur often enough in evaluation exercises. Many ToR tend to be rather outspoken either in favor of a qualitative evaluation approach or a quantitative evaluation approach, and the same is true for evaluators. Furthermore, an evaluation design including both extensive qualitative and quantitative research tend to be more expensive.

However, intermediate solutions exist. First: a proper quantitative evaluation should always be based on a qualitative pre-investigation. Otherwise the risk is (very) high that questions do not really cover the relevant issues, or these are not properly formulated. Secondly: in a primarily quantitative evaluation there often an opportunity to include qualitative inputs, either by including ?open? questions and/or by including limited qualitative additional research.

Similarly, qualitative research should be started after a proper quantitative analysis on the main issues. How otherwise to select the unit for a case-study, for participative observation, etc ? Furthermore, it makes more sense to place observations in a general context: expenditure of a family against average expenditure available in a community, school attendance of a village against general enrolment figures, etc.

Summarising: qualitative and quantitative evaluations serve different purposes. There is little sense in advocating either one of them as each has its own advantages and disadvantages. In general a mix is possible and desirable, taking into account the ToR and the qualities of the evaluator(s) concerned and the resources available