Rigorous, consistent, and agile evaluation of MacArthur’s work is a critical tool for informing our decision-making, leading to more effective stewardship of the Foundation’s resources and better results. We develop customized evaluation designs for each of MacArthur’s programs based on the problem, opportunity, and approach to the work, as well as Foundation priorities around outcomes, impact, and learning.
Evaluation is incorporated into each aspect of our programming – from when we first develop a program’s strategy, to implementing that strategy, to making any adjustments to it, and – finally – when we decide to bring an area of work to a close. This approach is anchored in four types of evaluation efforts, each of which are employed throughout the life of a program:
The four types of evaluation are:
- Landscape evaluation activities
- Feedback evaluation activities
- Performance and outcomes evaluation
- Impact evaluation
At the broadest level, we engage in landscape evaluation activities to gather information to better understand the context and environment of our work and how variables in the landscape may guide and inform our choices. We aim to be adaptive to evolving contexts and environments over time. Our primary questions are: What is the context, what are the social and environmental conditions, and how are they changing over time and space? Which external variables pose a threat and to which must we be sensitive?
More directly related to our programmatic work, we focus our evaluation around feedback evaluation activities, inclusive of reflection and feedback loops, to inform our relationships with our partners and grantees and the communications and supports provided. We believe that our work can be improved by hearing directly from those we engage with most directly (our partners, our grantees). Our primary questions are: What are the inputs , activities, and outputs associated with the program or project? What is working? What is not working and why? What could be improved?
Further, we use performance and outcomes evaluation activities to measure the results of our work. These measurement activities are designed to be carried out on-the-ground, as close to our funding and support as possible (e.g., a cluster of grants, a partner, or a grantee). Often these evaluations entail considerable field work. Our primary questions are: To what extent is progress being made toward specified goals? What are the measureable outcomes?
Over time, we use impact evaluation to better understand the ultimate contributions of our work. Our primary questions are: To what extent have the goals been achieved? What contributions has the program made? With what intended and unintended results?
Planning for an evaluation begins when a program strategy is first being developed and it is carried out in close partnership with program staff, and sometimes in consultation with external experts, throughout the duration of the program. Establishing evaluation activities at the outset of a program enables us to continuously adapt in response to the environmental context in which we work and what we learn along the way.
MacArthur is committed to sharing information about its evaluations both internally and externally in order to inform the future of our programs and as a way of advancing knowledge, understanding, and collaboration in fields in which we are engaged.