Abstract
Recent evaluations have almost always found Swedish development assistance relevant. Yet, beyond celebrating success, that common finding has limited utility, supporting neither Sida’s results-based management nor partners’ improved implementation. That warrants a fresh look and a critical exploration of the relevance criterion in evaluations of development cooperation, its use, and its assessment. The foundation for that exploration includes recent evaluations undertaken by and for Sida, sample evaluations for other agencies, OECD and Sida evaluation guidance, and evaluation research.
The review of the relevance criterion begins with Sida’s specification:
“The extent to which the intervention objectives and design respond to beneficiaries’ global, country, and partner/institution needs, policies, and priorities, and continue to do so if circumstances change.”
That specification itself creates several immediate application challenges. First, Sida, understands relevance as “doing the right things.” Are evaluators to determine whether evaluated activities are positively related (relevant) or correct (doing the right things)? Who determines what is right is left unaddressed, as is how to proceed when different notions of what is right diverge. A second problem is the assumption that diverse needs, policies, and priorities are complementary and consistent. In practice, often they clash. The criterion and its associated guidance do not recognize those tensions or provide adequate direction for evaluators on incorporating them in their assessments. Third, addressing all of the dimensions specified in the relevance criterion is beyond the reach of even the most extensive evaluation. In the absence of further guidance, relevance is too vast, too complex, and too conflicted to assess.
That the relevance criterion is a gatekeeper – if an activity is not relevant, why examine whether the wrong thing has been done efficiently? – encourages rapid, formulaic and superficial assessments. How have evaluators responded?