Education professor uses meta-analysis to help improve lives

Emily Tanner-Smith, Thomson Professor in the University of Oregon’s College of Education, grew up in the rural South in the 1990s. Within her small hometown, poverty birthed an almost certain future of teenage pregnancy, drug addiction or alcoholism for her classmates and their families.

Hope was not often on the menu.

“Because I grew up in this very dysfunctional environment, I always wanted to make a change in the world, and I wanted my scholarship to be a force for good,” Tanner-Smith said. “I really wanted to study how to prevent these things from happening in future generations. “

That desire incubated an intention that matured in her last few years of graduate school during a class at Vanderbilt University, where she received her doctorate in sociology in 2009. Her professor, Mark Lipsey, widely considered one of the founders of meta-analysis in the social sciences, exposed her to a research method that combines data from multiple studies on the same subject in order to draw conclusions from the entire body of scientific evidence.

The process shifts the potential impact of studies about a particular subject from a finding of an individual research study to an evidence-based framework on a particular topic that helps government strategically fund programs and set policies that lead to particular outcomes.

“It was just a major ‘aha’ moment in my career,” Tanner-Smith said.

After completing her doctorate, Tanner-Smith accepted a position at the Peabody Research Institute, which at the time was led by Lipsey, so she could work on a range of meta-analysis projects, which requires a depth of understanding about various research methods. Through those collaborations, Tanner-Smith had the opportunity to present and train researchers on meta-analysis across the world, including in South Africa, Denmark, Ireland and Norway. She worked at the institute until she was hired at UO.

“Emily was exceptional from the very beginning,” Lipsey said. “She is also one of the most effective and conscientious mentors I’ve ever seen. She quickly became a leader on the team and started initiating her own projects …. She left a big hole in our research team when she left.”

Meta-analysis has advanced Tanner-Smith’s research interests around what makes some young people resilient, how racial and structural inequalities influence adolescents’ outcomes, and what programs and policies can be used to promote positive youth development. Her work uses meta-analysis to understand what types of intervention and treatment programs are effective for reducing adolescent substance use and addiction, with specific focus on understanding variability in the effectiveness of interventions by context, setting and population.

“She has become one of the leading scholars in pulling that research together,” Lipsey said. “It’s important to be clear, this is the kind of work that’s intended to inform practice and policy.”

Early in her career, Tanner-Smith became a member of the Society for Research Synthesis Methodology, which Lipsey described as a “fairly elite organization that is heavy with senior people in the field and is invitational only.” Tanner-Smith’s recent meta-analysis work was used to inform the federal Office of Juvenile Justice and Delinquency Prevention’s evidence-based policy guidelines around best practices for juvenile drug treatment courts.

The guidelines provide juvenile courts with clear guidelines around how to promote family engagement to support youth with substance use disorders who are involved in the juvenile justice system. They are now being used in hundreds of juvenile drug courts across the United States.

Tanner-Smith also supports the U.S. Department of Education’s research synthesis work through their What Works Clearinghouse, which reviews existing research on education-focused programs, products, practices and policies. Tanner-Smith provides the clearinghouse with methodological guidance for conducting evidence reviews and oversees a system of reviewer training and certification.

In recognition of these and other meta-analysis contributions, Tanner-Smith has been received numerous awards, including the Nan Tobler Award from the Society for Prevention Research and the Robert Boruch Award from the Campbell Collaboration.

“She’s continued to do good work and get appropriate recognition for it,” Lipsey said. “This is work that has impact and essentially gets us into the next generation of interpreting research into practice and policy.”

Over the years, meta-analytic research has been embraced by many state and federal government agencies interested in evidence-based decision-making. The method received particular attention after a meta-analysis study published by the Campbell Collaboration contributed to the demise of a formerly federally funded program, Scared Straight, which was established in the 1970s.

The program brought young people, ages 14-20, who were connected with the juvenile criminal justice system into supervised contact with inmates in federal prisons. The intent was to expose youth to realistic depictions of life behind bars in an attempt to discourage them from breaking the law. The program was also the topic of a popular television documentary.

Meta-analysis researchers, however, discovered that Scared Straight provoked rather than prevented teenage delinquency, and it subsequently lost its federal funding.

“Increasingly, meta-analyses are considered the gold standard for informing evidence-based policy or practice decisions,” said Tanner-Smith, who teaches two meta-analysis courses in the UO’s College of Education. “But those meta-analyses must be conducted in a principled, transparent and rigorous way,”

Last year, Nicholas Parr was one of the UO’s first prevention-science doctoral students to graduate with a meta-analysis specialty after studying under Tanner-Smith. He accepted a position at the Veterans Affairs Medical Center in Portland, where he works in the VA Evidence Synthesis Program, which was established in 2007.

The center helps doctors, managers and policy makers improve the health and health care of veterans by tracking the validity of research and using it to shape evidence-based clinical policies and practices for veterans.

Parr credits Tanner-Smith with generously mentoring him as a doctoral student but also generously providing space for him to facilitate ownership over his professional and research trajectory. He said she also created opportunities for him to actually work on federally funded meta-analysis projects, and she served on Parr’s dissertation committee.

“She is a great mentor, very encouraging and motivating,” Parr said. “She views her role as cultivating your interest, motivation and enthusiasm for the field and less so about directing you to do this, that or the other thing. She wants to cultivate people’s motivation to do good science.”

Tanner-Smith said she enjoys creating space for students who show interest in the field to access opportunities to learn and grow in the specialties that interest them, and eventually further populate and expand the meta-analysis field, as her mentors did for her.

“To me, he is my first meta-analysis progeny from the UO,” Tanner-Smith said about Parr. “I’m just so proud of the work that he’s doing, using these skills to improve human health and well-being.”

—By S. Renee Mitchell, College of Education