Meet Julie Elworth, ITHS Director of Evaluation

19 Oct Meet Julie Elworth, ITHS Director of Evaluation

Julie Elworth is the new Director of Evaluation for the Institute of Translational Health Sciences. She works closely with staff and faculty to evaluate approaches to training and research implementation. Julie is part of the evaluation team which collects and analyzes the key data necessary to improve day-to-day operations, identify short-term and long-term changes, and help gauge the broader ITHS impact towards speeding science to the clinic.

We sat down with Julie Elworth to learn more about the skills and experience she brings to ITHS and how to incorporate evaluation methods into our everyday efforts. It was a Monday morning when we sat down to speak, Julie had spent the weekend breaking in the new ping-pong table with an intense family tournament, and was eager to share with me the ways in which evaluation can engage all the different areas of an organization and lead to substantial program improvements.

Tell us about your history developing evaluation plans for other organizations.

I am a sociologist by training and have worked in and studied areas from social psychology and small group analysis to populations of organizations. As a sociologist I’ve collaborated on research in attitudinal and behavioral aspects of energy conservation, meta-analysis of alcohol treatment programs, best practices for drug abuse and suicide prevention. I have also taught research methods to PhD candidates. I began doing evaluation full time in 2009 evaluating STEM education programs (Science, Technology, Engineering and Medicine).

What initially drew you to the field of evaluation?

I made the transition to becoming a full time evaluator because I was attracted to the applied component. Some of my previous work had been theoretical in nature whereas evaluation has a practical focus. Similar to the mission of the ITHS, evaluation is in a way its own ‘translational science’ from testing theories or ideas to application of those ideas in the field and in tangible programmatic changes.

What do you find exciting about working with ITHS?

At ITHS, there are a number of great projects that are all interrelated. It creates a very interesting environment for an evaluator. I enjoy getting to know people and getting to know their projects. This is what they have invested their professional time in; the love of their life really and their enthusiasm is infectious. The existence of an in-house evaluator is a testament to how important evaluation is to making sure each these projects succeed. I am excited to join the effort!

What are some of the challenges in program evaluation?

One of the main challenges in evaluation is ensuring that the project team knows I am on their side! Many times, evaluators come in late in the game or as a requirement from the funder which can provide a bit of tension for the team. My experience so far with ITHS teams is that they know that I am here to help ensure each project’s success from the beginning; not as an ‘add-on.’ I would like to continue to develop and foster the existing culture at ITHS so people see the evaluator not as someone looking over their shoulder, but as a partner in their activities to help collect information that will identify where things might need extra attention, adjustment, or changing.

An evaluator can offer a unique perspective about what type of information the funder would find valuable or that other folks may find beneficial. Funders do not expect to see a perfect project, they want to see a flexible plan that builds in strategies for tracking and identifying challenges, proposing solutions, and implementing changes along the way. After which one can own the successes, and help others learn from those successes; I have also discovered that the evaluator can actually highlight successes that the investigators do not always see or may take for granted.

Highlighting success sounds great! What are some other benefits of Evaluation?

In the simplest of terms, the evaluator is a second set of eyes. It is a service to the project team to have someone help measure how well the project is going and whether its reaching its outcomes. The evaluator can track all those pieces and help test them in very specific ways to ascertain that each one is functioning as it should and ensure that everyone is aware of the relationship between the project activities and the outcomes.

One of the things I have learned in this role is that it is not always clear to an investigator how an activity will lead to a desired outcome. That issue is not one I take for granted anymore. It is an important question to ask in order to make sure that both, the project team and I understand how all the pieces fit together. More often than not the essential project pieces are in the lead’s head, just not yet articulated or put down on paper. One of the things that the evaluator can bring to the table is a structure that by virtue of the questions posed, highlights the different parts of the project model.

What are some evaluation basics that people can easily incorporate into their work?

One critical piece of evaluation that everyone should incorporate into their work is to identify the assumptions being made about a project and use those assumptions to understand what pieces are likely to go awry. The ‘assumption’ process can be playful: what are the results when this happens? What does that say about our assumptions here? One of the greatest sources for identifying assumptions is new team members. A new team member is going to naturally ask questions as they learn about the project and they can be excellent sources for identifying assumptions that the project team – being so familiar with the idea – had lost track of. Seeking advice from these new team members, and talking with them about the project can benefit this process.

Another thing every team can do is identify steps in the project; what steps must be followed in order to implement an activity? When looking at individual steps, certain pieces must be adhered to or else the project will fail, no matter how small they are.

When is the best time to start evaluating one’s work?

When writing the proposal! Funders are requiring now that evaluators be brought in before the project is funded to help with the project structure, development, outcomes, and metrics. This articulation can also help with the proposal itself.

There is always a new opportunity to evaluate one’s work – just like the ping pong table, each hit provides new information and a chance to improve the next one.