Regardless of intent, foundation leaders can substantially harm or strengthen evaluation capacity building in their organizations. We can add value if we know how and why.
By Albertina (Aly) Lopez, PhD
Before I started my dream job at the Center for Evaluation Innovation (CEI), I worked with Kaiser Permanente’s Community Health as an internal evaluator and studied evaluation, organizational learning and change, leadership, and philanthropy at Claremont Graduate University. As I dug deep into the intersections of these fields, I found the sweet spot for my dissertation research: How can we (as leaders in foundations or consultants who work with them) build foundation capacity to do and use evaluation effectively?
Organizational evaluation capacity has two main elements:
- Capacity to do—the competencies and structures required to conduct high-quality evaluation studies.
- Capacity to use—the organization’s ability to integrate evaluation findings into its decision making.
Foundations have long grappled with how to ensure evaluation adds value to their work. My proposition is that foundation staff charged with this challenge can better meet it by defining how they want the organization’s evaluation capacity to look and by getting foundation leaders to influence the organization’s ability to achieve it.
My findings offer practical guidance. When building organizational evaluation capacity, we need to:
- Plan for how foundation leaders (evaluation, program, C-suite, board) can influence our goals.
- Build the evaluation capacity of foundation leaders.
- Factor in organizational context.
- Monitor, evaluate, and learn about the influence of leaders on our efforts.
To generate these findings, I developed an evidenced-based theory of change for how leaders affect foundation evaluation capacity building (ECB), and tested it using a mixed-methods approach.
The “ECB Leadership Framework” that resulted has four main concepts that really matter to the success of evaluation capacity building: ECB leadership, ECB followership, ECB goals, and organizational and environmental influencers.
The framework breaks these interdependent concepts down to identify the specific behaviors of leaders and followers that can facilitate or prevent the achievement of ECB goals.
1. ECB leadership is key. Leaders create the authorizing environment required to build evaluation capacity. They also influence whether others will participate in and sustain it.
Leaders drive and sustain ECB when they visibly make evaluation an organizational priority. Leaders must communicate clearly why evaluation and evaluation capacity building is important and engage people in the process so that they are aligned with the vision. ECB leadership is expressed by identifying evaluation capacity strengths and needs; planning out an evaluation capacity vision and ECB approach; and implementing that approach through organizational change or education.
Leader behaviors influence how other people in the organization perceive ECB’s value and whether they treat it as a priority, which I turn to next.
2. ECB followership represents staff intentions and actions in evaluation capacity building that come from interacting with leaders.
ECB followership includes people’s commitment—the mindset that binds them to contribute to ECB success or not, and their behaviors—the actions people take, from active and passive resistance to compliance, cooperation, and championing. We know from others’ research and mine that intention is a key antecedent to behavior and that this relationship is a causal loop where behavior affects intention.
Staff commitment and behaviors then affect the foundation’s ability to achieve ECB goals.
3. ECB goals are the intended outcomes for individuals, groups, and the organization that result from ECB participation.
Individual and group goals are getting people to have sufficient evaluation knowledge, skills, and attitudes. I emphasize sufficient because people in organizations do not need to know everything about evaluation, just the things that are valuable in supporting their work.
Organizational goals are achieved with an organizational design that effectively integrates doing and using evaluation. This can mean having an evaluation strategy that aligns with the organization’s values and mission; an organizational structure where evaluation has power and influence; processes and rewards for staff doing or using evaluation; or people with relevant and applicable evaluation knowledge, skills, and attitudes.
We know from the literature that there is a feedforward and feedback process in organizational learning, which explains the causal loop in ECB goals. Evaluation knowledge moves from individuals and groups (e.g., knowing how to develop an RFP for evaluation services) to the organizational level where it is institutionalized (e.g., having a standard, clear process for creating an evaluation RFP) and then back down where people learn the “norms” on how things are done (e.g., new and existing members can learn about and implement the standard process).
All of these components—ECB leadership, followership, and goals—are shaped by organizational and environmental influencers.
4. Organizational and environmental factors affect the relationships among these variables.
Organizational factors include organization design, culture, size, life cycle, time, and resources.
- What is the organization’s history with evaluation? How can leaders build on or depart from it?
- What can be done with available resources (e.g., financial, staff, time)?
- How does evaluation fit into the organization’s design? How does doing and using evaluation fit into people’s roles and responsibilities?
- What does the learning culture look like? Are people afraid of making mistakes or speaking up? Is learning from failure encouraged?
- How can existing energy around evaluation or learning serve as a launchpad?
Environmental factors are how the broader philanthropic and nonprofit sector and evaluation field influence the ECB effort, as well as the foundation’s reputational and accountability considerations.
- How do sector trends and focus (e.g., strategic philanthropy, equity) influence how people in the organization think? What does this mean for the evaluation capacity needed?
- How are foundation leaders being trained? Are using evidence and understanding data part of it?
- How will better evaluation capacity influence the organization’s reputation?
- Who is the organization accountable to? What kind of evaluation capacity is needed to meet it?
My hope is that the ECB Leadership Framework can be a tool that foundation leaders and consultants use to tailor ECB efforts so that evaluation is a valuable asset to the organization and its mission.
See how this framework comes to life and works in real-life foundations. Join me with Tanya Beer (Associate Director at CEI), Yvonne Belanger (Director of Learning & Evaluation at the Barr Foundation), and Matthew Carr (Director of Evaluation at the Ewing and Marion Kauffman Foundation) at the American Evaluation Association conference in Minneapolis on Friday, November 15 at 2:15 CT for our session, Leaders’ Influence in Evaluation Capacity Building: A Framework and Strategies for Cultivating Leader Commitment and Action.
 Lopez, A. A. (2018). Leadership in Evaluation Capacity Building: Influencing Organizational Change and Learning About Evaluation (Doctoral dissertation, The Claremont Graduate University).
 Bourgeois, I., Whynot, J., & Thériault, É. (2015). Application of an organizational evaluation capacity self-assessment instrument to different organizations: Similarities and lessons learned. Evaluation and Program Planning, 50, 47-55. In my ECB Leadership Framework, I break out organizational evaluation capacity at the individual, group, and organizational levels under ECB Goals because organizations learn by transferring knowledge across these levels.
 The Theory of Planned Behavior states that intention (like ECB commitment) is an antecedent to behavior and includes subjective norm, attitude, and perceived behavioral control (Ajzen, 1991). Cousins, Goh, and Clark’s (2006) notion of data use leads to data valuing can help us understand the relationship from ECB behavior to commitment: the researchers found that those who used data were more likely to value that data.
– Ajzen, I. (1991). The theory of planned behavior. Organizational behavior and human decision processes, 50(2), 179-211.
– Cousins, J. B., Goh, S. C., & Clark, S. (2006). Data use leads to data valuing: Evaluative inquiry for school decision making. Leadership and Policy in Schools, 5(2), 155-176.
 Crossan, M. M., Lane, H. W., & White, R. E. (1999). An organizational learning framework: From intuition to institution. Academy of management review, 24(3), 522-537.