Learn to critically appraise Randomized Controlled Trials and Meta-Analyses: Focus on studies examining the effects of exercise on health
Panteleimon Ekkekakis, Ph.D.
Professor, Department of Kinesiology, Iowa State University
Fellow, American College of Sports Medicine
Fellow, National Academy of Kinesiology of the United States of America
In clinical practice guidelines issued by medical and public-health organizations in several countries, exercise and physical activity are now listed among the recommended options for the prevention and treatment of a wide range of diseases and types of disability. What is perhaps not yet fully appreciated is that attempting to introduce a non-commercial option as a replacement for highly profitable commercial treatments has the potential to introduce considerable bias in the clinical research literature. This pre-conference workshop will focus on specific examples from the literature examining the effects of exercise on health.
The workshop is designed primarily for postgraduate students (Master’s and Ph.D. level), junior researchers, and public-health professionals. The workshop is not intended as an accelerated course in research methods or statistics. Therefore, some basic knowledge of research concepts is assumed (e.g., internal and external validity, Type I and Type II errors of statistical inference, statistical power, measurement reliability and validity).
By the conclusion of the workshop, participants will achieve the following learning objectives: (1) develop an appreciation for the prevalence of bias in clinical research, (2) understand the influence of social, economic, and political circumstances on the research process, and (3) sharpen their skills for evaluating research conclusions by learning to focus on aspects of research methodology that typically exhibit high susceptibility to bias.
The following is the outline of the workshop (coverage will depend on time)
- Ioannidis (2005): “Most published research findings are false”
- Basics of Evidence-Based Medicine (EBM)
- Critical appraisal as the weakest link of EBM
- Randomized Controlled Trials (RCTs)
a. Step 1: Learn about the political, social, economic, and policy context
b. Step 2: Clarify the implications of the research design
c. Step 3: Scrutinize the characteristics and the number of participants
d. Step 4: Understand the measures
e. Step 5: Critically evaluate the intervention and control/comparator treatments
f. Step 6: Compare the analytic approach to the published protocol (if any)
g. Step 7: Look for spin in the interpretation of the results
- Meta-analyses (MAs)
a. Step 1: Never take the integrity of data entry for granted
b. Step 2: Critically evaluate study inclusion and exclusion criteria
c. Step 3: Scrutinize the choice of « treatment » and « control » arms
d. Step 4: Learn about the implications of the fixed versus random model
e. Step 5: Understand heterogeneity and publication bias
f. Step 6: Closely examine the criteria of methodological quality
g. Step 7: Look for spin in the interpretation of results