A Brief Guide to Evaluation (Part 1)

This is the first in a series of blog posts, that outlines the key elements of an evaluation.

There are many things in the world that can be evaluated. You will want to decide and define the specific object (i.e. the “evaluand”) of the evaluation. One useful approach is for program managers, and program implementers to speak with the evaluator, and together, consider these defining questions about what needs to be evaluated:

Whether you are managing a program, funding an initiative, or about to undertake a policy review, you will want to first consider:

What is the “it” that’s being evaluated?

• program
• initiative
• organization
• set of processes or relationships
• campaign
• services
• activities
• product

Once you’ve identified what is being evaluated, it will be useful to consider these additional questions:

• What’s being done? What are the intentional actions that are being undertaken?
• Who does what to whom, and when do they do it?
• Why—for what set of reasons—are these things being done? Describe the need for the initiative or program?
• Who (which people, and which positions) are doing/carrying out the of the program, initiative, policy?
• What resources (i.e. inputs) are involved? (not just money and time, but knowledge, cooperation of others, networks of collaborators, etc.)
• Who (or what) are implementers working to change? What specifically is supposed to change, or be different as a result of the program or initiative doing what it does?
• Describe who benefits (e.g. program participants, beneficiaries, consumers, etc.) and what the benefits are.

By clearly identifying a determinate entity (i.e., “evaluand”) and the effects (i.e., outcomes) of the operation or implementation of that entity, you will help to ensure a successful evaluation. The clearer you can be in describing what the program, initiative, or policy is, the more effective the evaluation will be. Answering the above questions and sketching out a logic model of the program’s operation (which we’ll discuss in a subsequent blog post), will ensure an effective, accurate, and ultimately highly useful evaluation.

Also, for purposes of clarity, it may be useful for program managers and implementers to consider and discuss with the evaluator what isn’t being evaluated. This conversation can help to define the necessary boundaries around the program and will help to prevent evaluators expending unnecessary and potentially costly efforts to design evaluations of things that exist outside the boundary of the initiative, program or policy. To learn more about our evaluation practices visit our Impact & Assessment page.