Keeping on Track
Aligning goals, objectives, and
outcomes in evaluation design
Effective evaluation requires ensuring that goals, objectives, outcomes, evaluation design, approach to data gathering and analysis are in alignment. These processes need to be working together cohesively to meet the stated objective and to measure the effectiveness of the strategy at meeting that objective. Maintaining alignment is not as simple as it sounds. It is common that through the everyday process of developing, implementing and evaluating strategies a strategy can ‘drift’ out of alignment despite the best intentions of the organizers.
CASE STUDY TO ILLUSTRATE HOW EVALUATIONS AND STRATEGIES CAN DRIFT AND BECOME MISALIGNED
Below is an example of the ways in which even good projects, with clear objectives, strategies and evaluation conducted by committed stakeholders who have the best of intentions, can drift out of alignment and lose effectiveness.
Two front-line workers from Student Services want to develop and implement a strategy to address how to operationalize their college’s sexual violence policy. Their objective was to take a participatory approach to refine, clarify and operationalize the policy with input from the campus community. As they engaged key stakeholders from administration, they were encouraged to develop an educational video for students. In order to keep the administration involved and supportive of the project they agreed. Throughout the process they consulted student groups about bystander intervention believing that this fulfilled the ‘participatory’ aspect of the approach. They then developed a video. At the last minute a professor was invited to develop some evaluation questions. ‘Research creep’ (like scope creep) resulted in large scale research agendas shaping both the questionnaire content and the method of data collection. Instead of evaluating whether the strategy was effective at achieving the stated objective of operationalizing policy, or if the video was effective at imparting information about bystander intervention, the questions evaluated the effectiveness of the strategy at increasing empathy, which was a particular research interest of the invited professor.
While this example may sound far-fetched it is actually very common. Projects become disconnected and fall out of alignment because stakeholders don’t have a comprehensive, coherent plan from the start, they drift away from the plan as it is implemented, or because of scope (research creep) as other research agendas begin to take over. This is why developing an evaluation plan that aligns with the project’s objectives and sticking with it is key to effective evaluation.