This critique on simulation design was written by Alice Gray, a PGY 4 in Emergency Medicine at The University of Toronto and 2017 SHRED [Simulation, Health Sciences, Resuscitation for the Emergency Department] Fellow.
Have you ever designed a simulation case for learners? If so, did you create your sim on a “cool case” that you saw? I think we have all been guilty of this; I know I have. Obviously a unique, interesting case should make for a good sim, right? And learning objectives can be created after the case creation?
Recently, during my Simulation, Health Sciences and Resuscitation in the ED fellowship (SHRED), I have come to discover some theory and methods behind the madness of creating sim cases. And I have pleasantly discovered that rather than making things more complicated, having an approach to sim creation can not only help to guide meaningful educational goals but also makes life a whole lot easier!
I find it helpful to think of sim development in the PRE-sim, DURING-sim, and POST-sim phases.
In a systematic review of simulation-based education, Issenberg et al, describe the 10 aspects of simulation interventions that lead to effective learning, which I will incorporate these the different phases of sim design.1
PRE-sim
Like many things, the bulk of the work and planning are required in the PRE phase.
When deciding to use sim or not as a learning tool, the first step should be to ask what modality is most appropriate based on the stated learning objectives?1 A one-sized fits all approach is not optimal for learning. This is stated well in a paper by Lioce et al about simulation design that the “modality is the platform of the experience”.2 For me, one of the most important things to take into consideration is the following: can the learning objectives be appropriately attained though simulation, and if so, what type of simulation? For example, if the goal is to learn about advanced airway adjuncts, this may be best suited by repetitive training on an airway mannequin or a focused task trainer. If the goal is to work through a difficult airway algorithm, perhaps learners should progress through cases requiring increasingly difficult airway management using immersive, full-scale simulation. You can try in-situ inter-professional team training to explore systems-based processes. Basically, a needs assessment is key. The paper by Lioce et al. describe guidelines when working through a needs assessment.2
Next, simulation should be integrated into an overall curriculum to provide the opportunity to engage in repetitive (deliberate) practice:1 Simulation in isolation may not produce effective sustainable results.3 An overall curriculum development, while time consuming to develop and implement, is a worthy task. Having one simulation build upon others may improve learning through spaced repetition, varying context, delivery and level of difficulty.
This can be difficult to achieve given constrained time, space and financial resources. Rather than repeat the same cases multiple times, Adler et al created cases that had overlapping themes; the content and learning objectives differed between the cases but they had similar outcome measures. 3 This strategy could be employed in curriculum design to enhance repeated exposure while limiting the number of total sessions required.
Effective programmatic design should facilitate individualized learning and provide clinical variation: 1 Lioce et al, refer to a needs assessment as the foundation for any well-designed simulation.2 Simulation has addressed certain competencies residents are supposed to master – airway, toxicology, trauma, pediatrics, etc – without seeking input a priori on the learning needs of the residents. It may be valuable to survey participants and design simulations based on perceived curriculum gaps or learning objectives or try to assess baseline knowledge with structured assessment techniques prior to designing cases and curricula. (NB: Such a project is currently underway, led by simulation investigators at Sunnybrook Hospital in Toronto).
Learners should have the opportunity to practice with increasing levels of difficulty:1 It is logical that learners at different stages of their training require different gradations of difficultly. Dr. Martin Kuuskne breaks down the development of simulation cases into their basic elements. He advocates for thinking of each sim objective in terms of both knowledge and cognitive process.4
The knowledge components can divided into the medical and critical resource management (CRM), or more preferably, non-technical skills. 5 Medical knowledge objectives are self-explanatory and should be based on the level of trainee. Non-technical skills objectives typically relate to team-based communication, leadership, resource utilization, situational awareness and problem solving.6 Kuuskne’s post makes the very salient point that we need to limit the number of objectives in both these domains as this can quickly overwhelm learners and decreased absorption of knowledge.
The cognitive processes objectives can also be developed with increasing complexity, depending on the level of the trainee.4 For example, at the lowest level of learning is “remembering” – describing, naming, repeating, etc. At the highest levels of learning is “creating” – formulate, integrate, modify, etc. A case could be made to involve senior learners in creating and implementing their own sim cases.
DURING-sim
As part of creating scripts and cases, case designers should try to anticipate learner actions and pitfalls. There will always be surprises and unexpected actions (a good reason to trial, beta test and revise before deploying). On EMSimCases.com, Kuuskne outlines his approach to creating the case progression, and how can it be standardized.6 The patient in the simulation has a set of definite states: i.e. the condition of the patient created by vital signs and their clinical status.6 We can think of progression to different states through learner modifiers and triggers: Modifiers are actions that make a change in the patient, whereas triggers are actions that changes the state of the patient. I found this terminology helpful when outlining case progression.
Simulation allows for standardization of learning in a controlled environment: 11 The truth of residency training is that even in the same program, residents will all have uniquely different experience. One resident ahead of me, at graduation, had taken part in 10 resuscitative thoracotomies. Many residents in the same class had not seen any. We cannot predict what walks through our doors but we can try to give residents the same baseline skills and knowledge to deal with whatever does.
POST-sim
Feedback is provided during the learning experience1 unless in an exam-type setting, where it should be given after. It is important again to note the necessity of limiting the number of learning objectives, so you have room for scripted and unscripted topics of conversation. Debriefing the case should be a breeze, as it should flow from the case objectives created at the beginning.
Going further than “the debrief” is the idea of how we evaluate the value of sim. To me, this is the most difficult and rarely done. Evaluation of each sim case should be sought from participants and stakeholders, in addition to the pilot testing. That information needs to be fed forward to make meaningful improvements in case design and implementation.
Outcomes or benchmarks should be clearly defined and measured. The randomized study by Adler et al created clearly defined critical rating checklists during the development and needs assessment of their sim cases. 3 They then tested each case twice on residents to get feedback.
In summary, although a “cool case” is always interesting, it doesn’t always make the best substrate for teaching and learning in the simulator. Thoughtful case creation for simulation needs to go beyond that, breaking down the design process into basic, known components and using a structured theory-based approach in order to achieve meaningful educational outcomes.
REFERENCES:
1 Issenberg et al. Features and uses of high-fidelity medical simulations that lead to effective learning: A BEME systematic review. Med Teach. 2005;27:10 –28.
2 Lioce et al. Standards of Best Practice: Simulation Standard IX: Simulation Design. Clinical Stimulation in Nursing. 2015;11:309-315.
3 Adler et al. Development and Evaluation of a Simulation-Based Pediatric Emergency Medicine Curriculum. Academic Medicine. 2009;84:935-941.
4 Kuuskne M. How to develop targeted simulation learning objectives – Part 1: The Theory. April 21, 2015 https://emsimcases.com/2015/04/21/how-to-develop-targeted-simulation-learning-objectives-part-1-the-theory/
5 Kuuskne M. How to develop targeted simulation learning objectives – Part 2: The Practice. June 15, 2015. https://emsimcases.com/2015/06/16/how-to-develop-targeted-simulation-learning-objectives-part-2-the-practice/
6 Kuuskne M. Case Progression: states, modifiers and triggers. May 19, 2015. https://emsimcases.com/2015/05/19/case-progression-states-modifiers-and-triggers/
EM Sim Cases 