Beyond Effective Educational Survey Design: Planning Ahead for Data Analysis and Dissemination

Course Overview

Surveys and assessment tools are common in both programmatic evaluation and medical education research. If designed and implemented using best practices they can yield informative, actionable, and potentially publishable results. However, there are a number of important considerations that if not addressed during the survey design process can derail otherwise well-designed surveys.

During this workshop, participants will first review the steps necessary for effective survey design, with an emphasis on linking these early decisions in the design process to subsequent reporting and dissemination considerations. Participants will also learn advantages and disadvantages of different types of survey questions and response scales. In addition, we will review several methods for administering survey instruments, reviewing the pros and cons of each, as well as note important considerations related to data quality, statistical analysis, and peer review and publication. Then, participants will spend the majority of the time in small groups completing two activities. First, participants will work together to critique examples of survey questions that highlight different issues in not planning ahead for data analysis. Second, participants will develop their own survey items to explore a construct of their choice and, working in small groups, will review each others' surveys using a rubric to provide feedback on the questions, considering how question structure could lead to potential limitations in subsequent data analysis. Finally, in a large group format, we will review key lessons learned from the activities.

Learning Objectives

Describe and apply key concepts and steps in designing effective educational survey instruments.
Apply these steps during the workshop to create a short educational survey.
Review important considerations of survey design and elementary data analysis when planning for reporting and dissemination of results.

Speakers

Temple A. Ratcliffe MD
University of Texas Health Science Center at San Antonio Joe R. and Teresa Lozano Long School of Medicine

Jeffrey LaRochelle MD, MPH
University of Central Florida College of Medicine

Nicholas S. Duca MD
Penn State Milton S Hershey Medical Center

Michael Kisielewski MA
Alliance for Academic Internal Medicine

Additional Information

Year Published: 2022 - AIMW