Research frequently has shown that internally useful evaluation is difficult to achieve unless a program is "evaluable." A culture that is open to and supports evaluation and accountability is a prerequisite for achieving evaluability. This teaching case explores the process for creating such a culture.
Teaching cases are factual stories of one foundation’s in-depth experiences related to evaluation and learning. Stories highlight important challenges that confront foundations in their evaluation work, and put readers in the role of decision makers who are confronted with problems and options for solutions as the story unfolds. This teaching case was produced for the Evaluation Roundtable, a network of evaluation and learning leaders in foundations.
This teaching case describes the changing roles of an evaluator as an initiative develops and the needs of the evaluation’s users evolve. It is the story of the Central Valley Partnership funded by the James Irvine Foundation headquartered in San Francisco.
The case’s focus is on the relationship between the external evaluator and foundation staff.
The evaluation’s focus was learning and improvement, and as the evaluation developed recommendations, the evaluator was commissioned to implement his own recommendations, a form of developmental evaluation.
The case explores the dynamics and challenges of such an arrangement and relationship.
Traditional program evaluation and planning frequently present a straightforward process: the evaluator or planner (1) identifies a program’s goals, (2) develops operationalized measures of goal attainment, and (3) collects data on the measures to evaluate the program’s degree of goal attainment.
This teaching case illustrates how “unstraightforward” the picture frequently is for multi-dimensional programs with goals that are complex and dynamic and which take time to become clear and evaluable. In such programs the roles of planner and evaluator often merge in thorny ways with those of an organizational development resource and advocate.