Evaluation is by nature a highly interactive practice. It involves experiences, relationships, and collaboration amongst multiple players—staff and leadership at the lead organization, participants in the programming, involved community organizations, funding partners, perhaps an external evaluator.
One of the best ways that we’ve found to build an initial framework for a successful and comprehensive program evaluation is to start by understanding the broader context of the programming, defining the roles and relationships of those involved, and getting clear about expectations. We do this by asking a lot of questions!
Our initial conversations with a potential client interested in a program evaluation (whether for public health, education, economic development, etc.) often have nothing to do with specific program activities or data collection methods. We start by getting a sense of the bigger picture at the lead organization through some of the following questions:
Perceptions of evaluation and organizational culture
|
- What previous evaluation experiences have you had?
- How has evaluation played out in this context in the past?
- Is this an organization where evaluation is routinely part of ongoing work?
|
Wants and needs of the lead organization
|
- What do you see as essential aspects and key characteristics of the program?
- To what extent are the program’s goals and objectives clearly specified? Are the program’s outcomes specified?
- What are the program’s activities? What do participants do? What does staff do?
- How do culture and politics, large and small, affect this program?
|
Purpose of the evaluation
|
- Why are you planning to conduct an evaluation at this time?
- Who might be interested in the results of this study? Who will actually use the results?
- Do you hope to teach people about the evaluation process through this study?
- Is this evaluation part of ongoing data collection at the organization?
|
Organizational setting
|
- What is the organization’s mission? Is there a strategic plan in place? How might these relate to the proposed evaluation?
- What underlying organizational needs or conflicts might affect the evaluation?
- If this evaluation is conducted to meet funding requirements, how can it also be useful to other people?
- What role do you see for the evaluator in this project?
- What are your expectations for stakeholder involvement?
|
Evaluation approach
|
- To what extent will this be an evaluator-directed, collaborative, or participant-directed study?
- Are there cultural issues or expectations to attend to in choosing the approach?
|
Once we’ve established that the context will be a viable setting for a program evaluation—and that our team will be a good fit—we’ll continue on with a whole host of questions to help us define the methods for data collection and analysis, data interpretation, and reporting.
Let us know if you have any questions of your own!
For more information on an interactive approach to evaluation, we recommend Interactive Evaluation Practice by Jean King and Laurie Stevahn (Sage Publications, 2013).
Author: Suzanna Elkin
Suzanna Elkin is a social science researcher and educator with over 15 years of experience in public health and education research, planning, and evaluation. She holds a BA in Community Health from Brown University and a PhD in Educational Leadership and Policy Studies from the University of Vermont. Read more about Suzanna
