Presentation Design: A Picture is Worth 1000 Words & Better Evaluations?

Background: Lectures are the most common education form at medical conferences. For simplicity sake, generally, the lecturer stands in front of a group of learners and disseminates information via the use of slides. Learning theories, such as Mayer’s cognitive theory of multimedia learning, have shown that learners can either process visual or auditory information but not both simultaneously. Additionally, this processing is finite. One of the key lessons in presentation design, from psychological theories of learning, is to replace text with visual representations while the presenter tells a story.  This has also been shown to help improve retention of knowledge [2,3].  Most of the studies that show this are supported by medical student classroom learning environments. Little research has been performed in adult learning environments.

Mayer’s Cognitive Theory of Multimedia Learning

Multimedia Learning Principles

• Multimedia: Pictures + Words > Words Alone
• Modality: Spoken Text + Images > Written Words + Images
• Redundancy: Don’t Read Your Slides
• Contiguity: Keep Like Things Together (in time and space)
• Coherence: Remove Irrelevant Images & Words
• Segmenting: Break Lessons Down into Chunks
• Personalization: Conversational > Formal Speech

Article: Ferguson I et al. Continuing medical education speakers with high evaluation scores use more image-based slides. West J Emerg Med 2017; 18(1): 152-8. PMID: 28116029

What They Did:

• A retrospective analysis of six sequential CME conferences (2 annual emergency medicine conferences over a three-year period)
• Used a mixed linear regression model to assess the association of speaker evaluations with image fraction (i.e. percentage of image-based slides per presentation) & text density (i.e. number of words per slide)
• Lectures were evaluated on a five-point Likert scale (1 = Poor through 5 = Excellent) – Mean score of delivery, content, and practical value

Outcomes:

• Primary: Association of image fraction and text density with speaker evaluation scores

Inclusion:

• Videotaped lectures from the High Risk Emergency Medicine (HREM) Conference and The Topics in Emergency Medicine (TEM) Conference

Exclusion:

• 2 lectures by speakers involved in the design of the study (One study author lecture and one study collaborator lecture)
• Repeat lectures

Results:

• Important Study Statistics:
  • 105 unique lectures
  • 49 faculty members (Lectures given ranged from 1 – 8/conference – Median = 2)
  • 1,222 evaluations (70.1% evaluation rate)
  • Evaluators had a mean 14.9 years of clinical experience
  • Average image fraction = 47%
  • Mean text density (words per slide) = 25.61 +/- 8.14
  • Mean evaluation score for speakers = 4.5 +/- 0.24 out of 5.0
• Factors Associated with Higher Evaluation Scores:
  • The speaker
  • Speaker seniority
  • Higher slide image fraction
• Factors NOT Associated with Higher Evaluation Scores:
  • Total presentation time
  • Time per slide
  • Gender of speaker
  • Mean text density per slide
• Image fraction was the most likely factor to predict evaluation scores

Strengths:

• Evaluation percentage was within the normal range for most CME conferences (Average = 70.1% with a range of 41.1% – 85.6%)
• First study to assess the association of slide design and CME speaker evaluations by an audience of practicing clinicians

Limitations:

• Excluded title, disclosure, objective, and summary slides in the calculation of slide total which would decrease the overall number of slides to calculate the results
• Excluded words embedded in figures, slide headers, citations, and journal article screenshots from the final word count per slide which would underestimate the total number of words on a slide
• Evaluation scores were in a limited range of 3.5 – 5.0
• Speakers had a similar average number of words per slide (25.61 +/- 8.14) which may not have allowed adequate differentiation among presentations
• This study did not evaluate post-test knowledge retention (i.e. evaluation scores do not equal knowledge retention)
• No information on retention, learning of new information, or incorporation of information
• The primary outcome measure did not have a variable for slide design, making the final scores potentially confounded by lecture environment and presentation topic
• Only one author viewed and recorded data from all of the included presentations
• The regression model used in this study demonstrated an association with slide fraction and evaluation scores, but this does not equate to causation
• The difference in evaluation scores went from 4.40 – 4.60 on the graph meaning that the scores weren’t all that different. They did meet statistical significance, but not actually that different
• It is hard if not impossible to separate slide design from other things that faculty who understand multimedia concepts use in their teaching

Discussion:

• A great point brought up on twitter by Chris Nickson from the LITFL Blog

Author Conclusion: “This is the first published study to date assessing the linkage between slide design and CME speaker evaluations by an audience of practicing clinicians. The incorporation of images was associated with higher evaluation scores, in alignment with Mayer’s theory of multimedia learning. Contrary to this theory, however, text density showed no significant association, suggesting that these scores may be multifactorial. Professional development efforts should focus on teaching best practices in both slide design and presentation skills.”

Clinical Take Home Point: This is the first published evidence showing the association of increased slide image fraction to higher speaker evaluations.  Future studies will still need to assess the retention of short and long-term knowledge to evaluate effectiveness toward learning.

References:

1. Ferguson I et al. Continuing Medical Education Speakers with High Evaluation Scores Use More Image-Based Slides. WJEM 2017. PMID: 28116029
2. Issa N et al. Applying multimedia Design Principles Enhances Learning in Medical Education. Med Educ 2011. PMID: 21752078
3. Issa N et al. Teaching for Understanding in Medical Classrooms Using Multimedia Design Principles. Med Educ 2013. PMID: 23488758

For More Thoughts on This Topic Checkout:

• REBEL EM: Don’t Kill Your Audience with BulletPoints

Post Peer Reviewed By: Anand Swaminathan (Twitter @EMSwami)