Systematic Review and Meta-Analysis
Not currently scheduled | next offering date and format TBD
One of the greatest scientific challenges and opportunities in the information age is making sense and use of a vast sea of scholarly findings. Every time we write an introduction or acquaint or reacquaint ourselves with a field of study, we have to come to grips with the tsunami of articles that are all at our fingertips. By some estimates, scientific output doubles every nine years or so, but the hours in our days remains a constant twenty-four. Consequently, we all struggle with finding the relevant articles for our theses, introductions, meta-analyses and systematic reviews. Furthermore, we are expected often to search multiple databases, readily deduplicate them, keep them up to date, and do so all with transparency and replicability.
Once a set of articles that are relevant to one’s reviews have been relocated, the next challenge is making sense of the findings, which always vary from study to study. Reasons for such cross-study variation include differences in measurement and experimental procedures, stimuli, and rigor, populations sampled, time of study, and so forth. Meta-analysis is used to quantitatively extract the signal from the noise in this study-to-study variation in results as well as find systematic and reliable moderators of cross-study variation.
This course provides a theoretical and practical introduction to systematic reviews and meta-analysis. Using the latest machine learning algorithms incorporated into a cloud-based online platform, we can vastly reduce the time required to conduct a search and collect the data needed for a meta-analysis. Participants will learn how to conduct effective searches across multiple databases, upload them into the HubMeta platform for automatic deduplication, easily screen them using an advanced refinement of the PRISMA protocols (Preferred Reporting Items for Systematic Reviews and Meta-Analyses), annotate or tag them for later use, train RAs in the process, keep a record of their steps, and readily update their search. Whether the project is an individual’s thesis or a comprehensive meta-analysis with a large, international team, this course will make the process radically more replicable and easier. Time reduced can be over 90%, making this a necessity for anyone with limited resources.
INSTRUCTORS: Dr. Piers Steel, PhD and Dr. Hadi Fariborzi, PhD
In this course, you will learn about the underlying principles and the practical applications of systematic review and meta-analysis. The topics include
- Conducting effective searches
- Database selection
- Automatic article acquisition
- Establishing and training a research team
- Defining and conducting a literature search iteratively
- Citation chaining
- Deduplication and title/abstract screening
- Full text acquisition and screening
- Dealing with foreign language articles
- Tagging and sorting articles for later use
- Creating a taxonomy and connecting measures to constructs
- Converting effect sizes to a common metric
- The data entry process
- Dealing with dependent effect sizes and time series data
- Psychometric corrections for meta-analysis
- Meta-analytic models and their weighting schemes
- Outlier analysis
- Publication bias analysis
- Fundamental meta-analytic results
- Establishing potential moderators
- Meta-regression and moderator analysis
- Meta-analytic structural equation modeling
- Integrating other research techniques
- Open science reporting
This course is offered in online through Zoom as well as in person formats. Stay tuned to ccramsessions.com, join the mailing list for information about future offerings of this course, or click the "Registration interest" button to send us an email about your interest in this course..
The course will concentrate on practical application, with focus on data filtering, data entry and analyses in HubMeta and R, though referencing options in other software. Most of the provided statistical materials and examples will involve R code. HubMeta is a cloud-based platform that works best in the Chrome browser.
This course will be helpful for both introductory and advanced researchers in any field who are focusing on systematic review as well as mean-based or correlation-based meta-analysis. Experimental meta-analysis, though related, will be addressed only indirectly. Consequently, this course is relevant to researchers from most fields (e.g., psychology, management, sociology, education, human development, social work, public health, communication). Learners will ideally be comfortable with introductory statistics, including regression, and be able to read as well as understand the method section of basic articles in their field (i.e., necessary for data entry). Proficiency in R is desirable but not necessary. Learners should have access to academic search engines.
Upon completing both units of this course, you will
- be able to conduct a systematic review in an efficient, transparent, and replicable format
- be able to effectively construct search terms
- be able to train and supervise RAs for data screening
- configure machine learning algorithms for data screening
- effectively organize literature searches for later write-up
- be able to produce a competitive, tier-1 meta-analytic dataset, method section, and framework
- tackle meta-analytic topics an order of magnitude larger than previously plausible as well as in a fraction of the time, including comprehensive meta-analytic correlation matrices
- improve your management of a dispersed international meta-analytic research team
- address journals' theory requirements through meta-regression and meta-analytic structural equation modeling (MASEM)
- understanding the fundamentals of meta-analysis as well as the basic ideas behind more advanced techniques (e.g., One Stage MASEM)
A certificate of completion from the Canadian Centre for Research Analysis and Methods is provided at the end of the course.