AIR Webinar Series: Systematic Literature Reviews and Meta-Analytic Techniques

In recent years, science, technology, engineering, and mathematics (STEM) education researchers have used systematic review and meta-analysis methods to synthesize research evidence, identify gaps in the literature, develop instructional design guidelines, and inform teaching practice. Systematic literature reviews, however, are often confused with standard literature reviews. To reduce bias and increase transparency, a systematic literature review includes a precise research question with well-defined inclusion and exclusion criteria, formal literature search methods, and detailed study screening and coding protocols. In contrast, a standard literature review usually lacks formal review protocols and presents the author’s subjective interpretation of research on a given topic. After a systematic review, researchers often use formal statistical techniques known as meta-analysis to rigorously identify overall conclusions and variations in effect.

Join experts from the University of Oregon and the University of Louisville for a two-part webinar series designed to help researchers become better consumers of systematic reviews and meta-analyses. The series will provide an introduction to systematic reviews, highlight standards for top-quality reviews, and provide resources for learning about more advanced meta-analytic methods. Practical resources and applied STEM education examples will be emphasized. 

Webinar 1: An Introduction to Systematic Literature Reviews (held September 3, 2020)
Dr. Emily Tanner-Smith from the University of Oregon and Dr. Jeff Valentine from the University of Louisville will introduce the logic underlying systematic reviews. They will then share common challenges and provide practical advice for identifying the kinds of research questions that are suitable for systematic review, developing and assessing literature search strategies, and evaluating the credibility of the studies included in a review. Tools and resources will be shared to help the DRK-12 community apply those approaches.
Slides | Webinar Recording | Chat Archive

Webinar 2: An Introduction to Meta-Analytic Techniques (held September 28, 2020)
In the second part of this series, Drs. Tanner-Smith and Valentine will discuss how researchers can use meta-analytic procedures to combine quantitative data from multiple studies, statistically synthesize findings, and interpret meta-analytic results. They will discuss model choice in meta-analysis, describe the basics of computing and meta-analyzing study effects, illustrate how dependencies in effect sizes can be addressed, and discuss how publication bias tests are used (and misused).
Slides | Webinar Recording | Chat Archive


  • Emily Tanner-Smith, PhD, is a Thomson Professor in the College of Education and Associate Vice President for Research at the University of Oregon. She is an applied research methodologist with expertise in meta-analysis and research synthesis for evidence-based decision-making. She received the Nan Tobler Award from the Society for Prevention Research in honor of her research synthesis contributions to the field. She is a Features Editor for Research Synthesis Methods and an Editorial Board Member for Psychological Bulletin. Dr. Tanner-Smith previously served as a methods and statistical editor for Campbell Systematic Reviews and Systematic Reviews. Her current research is funded by the Institute of Education Sciences, the National Institutes of Health, and the National Institute of Justice.
  • Jeff Valentine, PhD, is a Professor in, and the Coordinator of, the Educational Psychology, Measurement, and Evaluation program in the College of Education and Human Development at the University of Louisville. He is a Principal Investigator (PI) for the U.S. Department of Education's What Works Clearinghouse. As an internationally known methodological expert, he has written more than four dozen works that use, explain, or seek to improve the methods of systematic review and meta-analysis. He co-edited (with Harris Cooper and Larry Hedges) the Handbook of Research Synthesis and Meta-Analysis, is an Associate Editor for Research Synthesis Methods, serves as a Statistical Editor for the Developmental, Psychosocial, and Learning Problems Group for the Cochrane Collaboration, and is co-chair of the Training Group for the Campbell Collaboration.

Objectives of the Webinars:

Following participation, audience members will:

  • Understand systematic review and meta-analysis terminology
  • Identify the importance and benefits of systematic reviews
  • Understand key considerations for the literature search, screening, and coding in systematic reviews
  • Understand key considerations when synthesizing evidence using meta-analysis
  • Consider ways systematic reviews and meta-analytic techniques could further new learning in STEM education

Questions? E-mail Sarah Rand at


These webinars are supported by a grant from the National Science Foundation (DRL-1813777). Opinions, findings, and conclusions or recommendations expressed do not necessarily reflect the views of the National Science Foundation.