Does Germany currently make use of existing evidence?

In an empirical research project DEval analysed the current state of rigorous impact evaluation (RIE) in German development cooperation (GDC). Here we answer questions, such as: How familiar are employees in Germany with rigorous evidence? How much rigorous evidence do they use in their daily work? What is the attitude of employees towards using evidence?

Below, you find 4 key findings:

Key finding 1: Knowledge about products based on rigorous evidence is rather limited among GDC employees: respondents in our evidence survey indicated that they were barely familiar with rigorous impact evaluations (RIEs), systematic reviews (SRs) and evidence gap maps (EGMs).

Figure: Knowledge about RIE, SR and EGM

RIE
7 %
13 %
23 %
23 %
31 %
4 %
SR
3 %
14 %
22 %
29 %
26 %
6 %
EGM
1 %
8 %
14 %
23 %
48 %
5 %
Already conducted one or more myself
(2)
(3)
(4)
Never heard about this before
Don't know

Source: evidence survey; Question: “How familiar are you with RIE, SR, EGM?”; Scale 1–5; N=839.

Further Information

Knowledge about RIE is a precondition for the take-up of rigorous evidence. When participants already have practical experience in conducting RIEs, SRs and EGMs this means they have profound knowledge. This is the case for only very few respondents (RIE: 7%, SR: 3%, EGM: 1%). About one fourth of the respondents have only heard of these evidence products before meaning that they have only little knowledge. A large share of the respondents had never heard of the products before (RIE: 31%, SR: 26%, EGM: 48%).

Key finding 2: Self-generated rigorous evidence is used in GDC to some extent. The potential of evidence from self-generated RIEs is however not fully exploited.

Further Information

Lessons from self-generated RIEs are often shared with other projects within GDC organisations. In some cases, they are also used for designing follow-up projects and for adjustments within the current project. Results are often use for accountability, mostly toward the Federal Ministry of Economic Cooperation and Development (BMZ). Self-generated evidence from RIEs is however not used for strategic decision-making.

„Look at the strategies; they are full of normative theses. But they don’t somehow say that this has proven itself. [...] There is no empirical foundation for such things. I hardly know of any strategy that says: We’ve had great experiences, they’re robust, and that’s why we’re going to continue. (Int_35).”

Find out what we mean by "self-generated"


We define self-generated rigorous evidence as findings from RIEs that have evaluated a specific project within GDC and which are used in that context.

Key finding 3: The global RIE evidence base is rarely used and very rarely explicitly referenced in project documents.

Further Information

Scientific sources in general, and sources describing rigorous evidence in particular, are rarely referenced in project documents.

In formal reporting documents, such as progress reports, claims about the causal impact of projects were frequently made but either supported by non-rigorous evidence or no evidence at all was presented.

 

Find out what we mean by "global"

We understand the global RIE evidence base to comprise RIEs that have been conducted by other projects within one’s own organisation or by other organisations globally, including development partners or other donor countries.

Source: evidence survey; question: “To what extent do you agree with the following statement? In order to make GDC more effective, I think the use of evidence is generally desirable”; N=1054; scale 1-5. Evidence survey; question: “To what extent do you agree with the following statement? For the quality of my daily work it would be helpful if I could use substantially more evidence”; N=907; scale 1-5.
Key finding 4: Despite the limited use of rigorous evidence in GDC, GDC employees have a strong demand for and a very positive attitude towards using evidence.

Figure: Attitude towards evidence take-up

Further Information

A clear majority of respondents (strongly) agreed that evidence take-up is key to making GDC more effective (94%). Moreover, about 75% of GDC employees stated that the use of evidence would substantially improve the quality of their own work. This high demand for evidence differs neither across organisations nor across different functions within an organisation. These findings speak for a strong demand by GDC professionals for an evidence-informed approach to development cooperation. But rather the opposite is the case when considering the actual use and visibility of evidence in the current formal procedures.

To Top