Measures of Improvement
Measures of Improvement
Throughout her career, EDC’s Leslie Goodyear has held a core belief: Evaluation is the key to unlocking insights into how more programs can serve more people, more effectively. In the past few years, she has focused on evaluating programs that aim to broaden the participation in STEM careers of women, people of color, and people with disabilities. Her experiences have helped inform her perspective on why it’s critical to bring more diversity to the evaluation field, as well.
“It’s important to me that the next generation of evaluators are more diverse and can better understand the populations, issues and contexts of the programs they evaluate,” says Goodyear, who has evaluated National Science Foundation-funded STEM education programs for a decade.
With her term as president-elect of the American Evaluation Association (AEA) set to begin in January 2017, Goodyear shares some thoughts about why evaluation is important and what she has learned about broadening participation in STEM.
Q: What do most people not know about evaluation?
Goodyear: Evaluation is about learning, reflecting, and making things better. I think some people see evaluation as a very stern, judgmental, impersonal process, but I don’t see it like that. The programs that I evaluate have a lot of knowledge about what works within the context of their field and in their communities.
As an evaluator, my job is more than measuring outcomes. It’s also helping those programs apply that practical knowledge in the service of programmatic improvement. There are lots of ways to do this, and I work in collaboration with program staff to develop the most appropriate evaluation plan.
Q: What have you learned from your work evaluating programs that are trying to broaden participation in STEM?
Goodyear: Programs have to be thoughtfully and purposefully designed, and they also have to make sure they are reaching the intended audiences. So imagine a STEM education program in an urban school. The population that the school is serving may largely be from groups who are underrepresented in STEM. But that doesn’t mean that the students who participate in the program are, for example, girls or students of color. It could be that the program is most appealing to boys who already have an interest in STEM. Evaluation helps programs know what’s really happening on the ground, who is being served, and just as important, who isn’t being served.
In addition, from my experience evaluating STEM education programs, I know that students need to be able to make connections from their STEM education experiences to their lives. It’s often not enough to drop, say, a robotics program into a school and expect it to make a huge difference in excitement about STEM careers. But if you design that robotics program with linkages to the classroom, to the afterschool program, and to the students’ home experiences and other interests, then there’s a chance that that program can make a big difference.
Q: Why is it important to broaden participation in the field of evaluation itself?
Goodyear: As a field, evaluation is most constructive when the evaluators themselves understand the contexts and cultures within which a program is occurring. And, like the STEM and social science disciplines, bringing more diverse backgrounds and perspectives to the field of evaluation opens the door to innovation. It’s important to harness this moment when there’s so much interest in evaluation—from funders and government—and use it to bring new voices to the field.
That’s why I think programs like the AEA’s Graduate Diversity Education Initiative are so important. A more diverse profession will simply make the entire field better.
Q: What evaluation advice would you offer to projects?
Goodyear: Engage your evaluator early and often! Evaluating includes measuring—and in order to measure something, you have to be really clear about what that something is. Evaluators are really good at asking questions that clarify that core idea, question, or purpose behind a program. Addressing these questions early can help make for a better program, too.