One core element of RIE is a control or comparison group against which the intervention group is compared (often called "counterfactual"). In an experimental design, participants are randomly assigned to the control or the intervention group before the start of the intervention. In a quasi-experimental design, statistical methods are employed to construct valid intervention and comparison groups (often retrospectively).

The required sample size depends on a number of factors, including the expected magnitude of the impact caused by an intervention and the number of different components of the program/project to be tested. A statistical power calculation helps to estimate the minimum sample size required to detect differences in outcomes between the intervention and comparison groups.

RIEs often require substantial financial resources, mostly because of high costs for primary data collections with large samples (e.g. wages of enumerators). The costs for RIEs typically start from USD 300.000 and can exceed USD 1 M. In cases where existing data, such as monitoring or administrative data, can be used, the costs of the RIE can substantially be reduced.

Access to data and the quality of the data depend on factors such as cooperation with local stakeholders or well-trained enumerators. Either secondary data can be used, such as monitoring or administrative data, or own primary data can be collected.

The successful implementation of an RIE requires both specialised technical RIE knowledge (e.g. knowledge of suitable RIE designs and their implementation and data analysis skills) and practical understanding of the sectoral and country-specific context.

Three temporal parameters are necessary to conduct an RIE. First, in the case of randomised controlled trials (RCTs), random allocation into control and intervention groups must take place before the start of the project intervention. Second, there must be a plausible amount of time between the end of the intervention and the collection of the data in order to be able to measure an impact. What is considered a “plausible amount of time” can vary widely, depending on the nature of the intervention and its intended outcomes, as reflected in the underlying theory of change. Third, in case of ongoing interventions, they should not be changed as long as the data collection is running to ensure that intervention outcomes are valid.

Can you assure the following temporal parameters?

The commitment of stakeholders (e.g. partners, project team, funder), their demand for and openness to an RIE is crucial for a smooth implementation of an RIE. This includes involving staff at all levels: from decision-makers at the management level to enumerators in the field. Further, considering the socio-political context (e.g. political stability, access to stakeholders etc.) helps to assess whether a successful RIE implementation can be expected.

There are a number of ethical considerations when conducting an RIE, given that human beings are the study subjects. In their Ethical Guidelines for Evaluation, the UN Evaluation Group defines four areas of ethical behaviour that must be complied with: integrity, accountability, respect, and beneficence (UNEG, 2020). Furthermore, a clearance process by an ethics committee or Institutional Review Board (IRB) can ascertain the ethical feasibility of the RIE.