Debriefing Techniques – the Art of Guided Reflection

Simulation without debriefing is really just an expensive way of either making learners feel badly about themselves or allowing learners to practice performing poorly. This is why the theory behind debriefing is so important.

Debriefing is one of the most amazing teaching tools available to an instructor. Debriefing allows insight into a learner’s thought process such that an instructor can tailor teaching to a learner’s specific needs. Kolb’s learning cycle1 and Schonn’s description of the Reflective Practitioner2 allow us to see why debriefing is such a useful tool. We must actively reflect on an experience to learn from it; debriefing allows educators to help guide that reflection.

PEARLS Framework

While debriefing is arguably the most important component of simulation education, it is also a difficult skill to acquire. Eppich and Cheng3 have published an excellent approach to debriefing that reviews many of the key steps a novice simulation educator should aim to follow. They have called it the PEARLS approach (Promoting Excellence and Reflective Learning in Simulation). We will review its four phases here.

1. Reactions Phase

This is where learners are invited to express their raw feelings about the case. Often, learners will do this without a formal invitation (for example, you may hear initial reactions while walking from the simulator to the debriefing room). It is important to invite all learners to have a chance to vent during this stage.

2. Description Phase

This phase begins by asking a learner to describe what they think the case was about. This allows the educator and the learners to see if they are on the same page. Often, this leads to important issues for discussion during the next phase.

Screen Shot 2015-06-28 at 1.15.24 PM

3. Analysis Phase

Here, the educator must tailor their style of debriefing to suit both the learners in the room and the time available for the debriefing. This phase is what educators often think about when they envision debriefing. Essentially, the analysis phase is where learners can go through guided reflection.

+/Δ Method

There are two common styles of guided reflection described. The first is the +/Δ method. This involves probing learners as to what went well (the +) and what could be improved or changed for the future (the Δ). Many who are new to debriefing find themselves turning to this style at first.

Advocacy/Inquiry Method

A second, commonly used style is called advocacy/inquiry.4 This approach leads to incredible insights into the knowledge and performance of the learners. It can be somewhat more challenging to execute well. The basic premise is that one must first describe a noted performance gap. This is followed by a question as to the learner’s frame of mind at the time of the performance. The learner’s answer leads the instructor as to what learning points may need to be addressed. Sometimes, the entire room of learners is unsure of a next appropriate step in management. In this case, the debriefer must simply provide directed teaching. In other cases, the learner has made a slight cognitive error. Often, these can be addressed through facilitated discussion with other learners.

4. Summary Phase

Once the group has gone through all the desired learning objectives in the analysis phase, it is imperative that the instructor guides a review of key points related to the objectives. If time is short, the instructor can provide the summary himself. If time is more abundant, it can be useful to have the learners go through their key learning points.

As we can see, a fair amount of effort is required to facilitate an excellent debrief. With frameworks like the PEARLS approach, experienced and inexperienced educators alike have a practical means upon which to build their debriefing skills.

What tips and tricks do you use in your debriefing?

References:

  1. Kolb DA. Experiential earning: experience as the source of learning and development. Englewood Cliffs, NJ: Prentice Hall; 1984.
  2. Schon D. The Reflective Practitioner: How Professionals Think in Practice. New York: Basic Books. 1983.
  3. Eppich, W., Cheng, A. Promoting excellence and reflective learning in simulation (PEARLS). Simul Healthc. 2015:1. doi:10.1097/SIH.0000000000000072.
  4. Rudolph, JW., Simon R., Rivard P., Dufresne RL., Raemer, DB. Debriefing with good judgment: combining rigorous feedback with genuine inquiry. Anesthesiol Clin. 2007;25(2):361-376.

Case progression: states, modifiers and triggers

In order for a simulated scenario to run smoothly, the case progression needs to be planned for in advance. This involves determining which states the patient simulator progresses through, how modifiers may change features of those states and what triggers will be used to change between states. A working understanding of these terms makes developing cases a lot easier.

State

During a simulated resuscitation scenario, the patient progresses through multiple states. The state represents the overall condition of the patient simulator during a specific period of time. I like to think of a state as a constellation composed of the vital signs and the patient status (which includes the general appearance and relevant physical exam findings) that we can present to the learners. While case progression usually follows a linear route through different states, this is not the rule; the case may skip or jump to a different state depending how it is developed (see figure 1). Each state should be represented by a characteristic title.

Screen Shot 2015-03-22 at 12.58.55 PM

Figure 1. An example of a case progression. States 1 through 4 represent a linear progression. State 5. V-fib, is a possible simulator state, depending on the leaners’ actions. The green arrows represent unspecified triggers.

Modifier

A modifier is a learner action that induces a change to the patient simulator, but not enough to transition between states. These changes can affect either a vital sign or a component of the patient’s status, but usually not both. An example of a modifier would be the application of a 100%-non-rebreather mask to a patient in an “Acute Pulmonary Edema” state. As a modifier, this learner action would cause an increase in the patient simulator’s O2 saturation from 84% to 89%. However the state, “Acute Pulmonary Edema”, would not change. It would continue to be represented by sinus tachycardia at a rate of 120, a blood pressure of 180/105, a respiratory rate of 28 and a patient status represented by respiratory distress (accessory muscle use, pursed lipped breathing etc). A modifier can manifest its change instantly or over a specified amount of time (ex. increase the O2 saturation from 84% to 89% over 10 seconds).

Trigger

A trigger is an event that causes a change in the simulator state. I describe triggers as being either active or latent. Active triggers are represented by a learner action (ex. needle thoracostomy) or a specific combination of learner actions (ex. ≥2 methods of active cooling) while latent triggers are usually time-based (ex. 3 minutes). Active triggers are key to the progression of the case and make for great learning points during debriefing because they define important medical management decisions. Latent triggers are used to automatically progress the case. Like a modifier, a trigger can also be manifested instantly or over a specified amount of time.

EMSimCases case progression template

Figure 2. An example of a state, modifier and triggers using the EMSIMCASES case progression template

Figure 2. An example of a state, modifier and triggers using the EMSIMCASES case progression template

The EMSimCases template uses a table to display and facilitate case progression while running a simulation scenario (see Figure 2). The patient state is described in the first column with its title and vital signs. The patient status (general appearance and relevant physical exam findings) is described in the second column. A full physical exam is described in another section of the template. The third column lists possible learner actions. The fourth column contains the modifiers and triggers for that state.


Any simulation educator can tell you that no matter how much planning goes into case development, learners will always surprise you with an action that you did not predict. This highlights the importance of being able to adapt the case progression to unforeseen learner actions on the fly. However, if you develop cases with a logical progression of states, account for possible modifiers and how they will change features of those states and, lastly, define the triggers that will transition between states, your simulation scenario will be as smooth and realistic as ever.

Realism

What is it?

Realism is the degree to which your simulation environment recreates or mimics the patient environment for your learners.

A word on fidelity.

The terms realism and fidelity are essentially interchangeable. However, many often associate the term fidelity with the amount of technology used to recreate the patient environment. For example, when educators refer to a case as “high fidelity” what they often mean is that they are using a costly computer-based mannequin with all the bells and whistles. The caveat, of course, is that having cutting edge equipment does not, on its own, ensure that the learner’s experience approaches reality. I prefer the term realism because it reminds us that there are more things to simulate than just the physical environment.

Why it’s important.

The basic premise of simulation as an educational modality is that it allows direct observation of a learner’s behaviour. Furthermore, debriefing in simulation allows discussion about noted learner deficiencies. Teasing out the learner’s cognitive process and knowledge gaps to discover the origins of the learner’s behaviour is paramount. In order to elicit true behaviour from a learner, (i.e. – behaviour that most closely mirrors their performance with real patients) the learner must treat the situation as a real one. And to do so, they must believe in it.

If the environment in which the learner is practising does not even come close to imitating reality, then the learner will not fully engage in the learning exercise. This limits the ability of the instructor to assess the learner’s abilities. In addition, not addressing realism lets learners use it as an excuse for their performance. For example, “If the mannequin had better breath sounds, I would have decompressed the tension pneumothorax.” Or “If this case was in the Emergency Department, I’m sure I would have seen the VT on the monitor and then shocked the patient.”

Making the environment mirror reality does not necessarily require high tech equipment. It does, however, require engaging the learners and addressing limitations to realism before the scenario begins. Orient learners to the mannequin so they know where they can feel pulses and where to listen for breath sounds. If the mannequin doesn’t have these things, let the learners know how to ask for physical exam findings. It is remarkable how well learners can engage in a scenario with a mannequin that has no high tech functions. They are only able to do this if you create conceptual realism.

Types of realism

In 2007, Rudolph, Simon, and Raemer described three different types of realism as essential to simulation training.1 Their terminology was a slight modification of Dieckmann’s work on the aspects of realism, also published in 2007. 2 The three components of realism highlighted by Rudolph et al are as follows:

1) Conceptual

Conceptual realism allows learners to think about a case in the same manner they would for a real patient. The most important component to creating conceptual realism is providing the learners with enough information to accurately frame the case. For example, you would expect a patient with a tension pneumothorax to have tachycardia, hypotension, and decreased breath sounds on one side. How this information is conveyed matters less than the fact that the information is logical in the context of the case.

To understand the power of conceptual realism, look to oral exams. The learner is able to make a diagnosis and manage a patient without any physical cues present. Oral exams can create conceptual realism. Conceptual realism is crucial to a good simulation scenario. And sometimes, adding too many bells and whistles actually takes away from the concept.

Yes, that’s right. You can be very low tech and still run fantastic simulation. You just need to set the stage, meet minimum cognitive standards, and debrief.

2) Physical

There are some things that just need to be practised in real time and space. Physical realism is most important for procedural skills. Practising airway management on an airway head that has unrealistic anatomy just doesn’t help learners to develop the motor memory they need. This doesn’t mean that all simulators need to be exact replicas. But to create physical realism, a task trainer must emulate the necessary motor feedback required to practise a skill properly. For example, a chest tube trainer doesn’t need to be an entire pig chest. It does, however, need to have an appropriate degree of resistance so that learners develop the sense of how hard to push in order to penetrate the pleura.

All mannequins have poor physical realism in some way. But with enough cognitive and experiential realism, it doesn’t need to affect the quality of the learning experience.

3) Emotional and experiential

This is the type of realism that puts a knot in your stomach. Experiential realism is about creating the emotions that often make our jobs difficult. Examples would include having a mother sob in the corner while trying to run a code on her infant child. Or having a difficult parent present who becomes obtrusive to care. Or how horrifying it can be to see a patient with a GI bleed exsanguinating from their mouth. Perhaps the challenge is creating the cognitive burden that goes along with managing two patients at once. Or perhaps the experiential realism comes from the frustration of dealing with a team that is obviously ignoring your direction. In other words, experiential realism is important to consider if the purpose of a case is to practice working through an emotionally challenging case or to teach techniques for overcoming a difficult family member or team member. It is also an important part of why junior learners can find simulation intimidating – because good experiential realism recreates the fear or discomfort that goes with being uncertain how to manage a particular condition. Again, your mannequin can be a cabbage patch kid doll if your sobbing parent actor is good enough.

The reality of realism

Realism is essential to simulation. As a simulation educator, you should be aware of which aspects of realism are most important for the case you are designing. Do you need to create an appropriate cognitive environment to assess the resident’s management of a TCA overdose? Do you need to see how the resident can lead a difficult team? Or do you need to see that a resident can skilfully perform a cricothyroidotomy? Or do you need all three components to assess a resident’s management of a pediatric trauma? Design your case and supplies with your realism goals in mind.

References

  1. Rudolph JW, Simon R, Raemer DB. Which reality matters? Questions on the path to high engagement in healthcare simulation. Simul Healthc. 2007;2(3):161-163. doi:10.1097/SIH.0b013e31813d1035.
  2. Dieckmann P, Gaba D, Rall M. Deepening the theoretical foundations of patient simulation as social practice. Simul Healthc. 2007;2(3):183-193. doi:10.1097/SIH.0b013e3180f637f5.

How to develop targeted simulation learning objectives – Part 1: The Theory

Screen Shot 2015-03-20 at 1.41.59 PM

Miller’s Triangle (adapted from “The assessment of clinical skills/competence/performance.”) 

Simulation has filled a void that was once present in medical education. Written and oral examinations continue to be used to assess Miller’s “knows” and “knows how” levels of performance while clinical rotation evaluations rest at the top of the triangle: “Does”. Simulation completes Miller’s triangle by allowing learners to “show how” their knowledge and skills can be applied in a risk-free, simulated clinical environment. 1-2

As simulation educators, our roles not only include creating, programming, running realistic scenarios and facilitating debriefing, but also developing appropriate learning objectives that align with our instructional strategy of simulated team-based resuscitation.

Learning objectives are statements of what we intend or expect students to learn as a result of our instruction. In order to create these objectives, we need to determine what kind of knowledge and cognitive processes we are trying to address in our learners through the use of simulation. This is where learning theories can help.


Learning Objective Taxonomy

Screen Shot 2015-03-20 at 1.45.51 PM

Krathwohl’s Knowledge Domain and its categories (adapted from “A Revision of Bloom’s Taxonomy: An Overview”)

Bloom originally described a hierarchical taxonomy of educational objectives based on 6 categories of the cognitive domain from simple to more complex: Knowledge, Comprehension, Application, Analysis, Synthesis and Evaluation.In 2002, Krathwohl presented a revision of Bloom’s Taxonomy that expanded and described the different categories of knowledge in increasing complexity from factual to metacognitive knowledge. He also described a novel approach to educational objectives involving two dimensions: a combination of the type of knowledge and the cognitive process involved in obtaining that knowledge.

This combination represents the way learning objectives are usually developed and written; there is a component of subject content as the noun (the knowledge domain) and a description of what is to be done with that content as the learning verb (the cognitive process dimension).

Screen Shot 2015-03-20 at 2.01.04 PM

Krathwohl’s Cognitive Process Dimensions are presented from the least to the most complex cognitive process (adapted from “A Revision of Bloom’s Taxonomy: An Overview”) 

An instructor can develop learning objectives for simulation that fall under any knowledge domain and cognitive process dimension but being a learning modality that can be limited by resources, cost, space and time, I believe that these learning objectives should be optimized to the most appropriate cognitive process dimension and should specifically target subject content from a complex knowledge domain so that learners get the most out of the simulated experience. Compare these 3 examples of learning objectives for a simulated scenario of unstable bradycardia.

Ex 1: Understand the treatment of a patient with unstable bradycardia.
Ex 2: Recall the appropriate dose of atropine in the setting of unstable bradycardia.
Ex 3: Appropriately employ the ACLS bradycardia algorithm to a patient in 3rd degree AV-block.

In example 1, the learning verb, to understand, is ill defined. A non-specific learning verb makes it more difficult to assess the learner’s performance. Also, while treatment as the noun may be classified as conceptual knowledge, it is too vague to tailor specific debriefing comments towards.

In example 2, the learning verb recall uses the lowest level of cognitive processes: remembering. Also, the dose of atropine represents factual knowledge, the lowest level of the knowledge domain. While this could be an objective for a simulated scenario, the objective could also be adequately (or even more appropriately) met using less complex instructional strategies (textbooks, blogs or lectures) or assessment tools (paper tests or oral exams).

In example 3, the learning verb employ targets the highest cognitive process dimension of the three examples: to apply. The ACLS bradycardia algorithm represents procedural knowledge that is well defined which helps both the learners and educators understand their expectations. The learning objective is specific and tailored to the case.

Which levels of knowledge and cognitive process dimensions should we target?

While learners in simulated scenarios do employ factual and conceptual knowledge in the evaluation, diagnosis and treatment of the simulated patient, I think the facets of procedural knowledge (remember, this is the educational theory “procedural”, not cricothyroidotomy “procedural”) best represent the kind of knowledge simulation can afford to learners. These facets include subject-specific skills, algorithms, techniques, methods and criteria for determining when to use appropriate procedures. Debriefing scenarios can also incorporate the metacognitive domain as learners can reflect on their performance and gaps in their knowledge. The apply dimension, which includes executing and implementing procedures in a given situation, most adequately describes the cognitive process used by learners during simulated scenarios while debriefing may involve evaluating certain processes.

Each cognitive domain process has useful learning verbs associated with them to help us create targeted learning objectives.

Screen Shot 2015-03-20 at 2.29.36 PM

Adapted from “A Revision of Bloom’s Taxonomy: An Overview”

Final Tips

So, when developing your learning objectives for a simulated scenario, targeting a specific knowledge domain and cognitive process as the learning noun and verb, respectively, will help guide the process. To optimize the learning objective and align it with the instructional strategy of simulated resuscitation scenarios, try and aim for a more complex knowledge domain (such as procedural knowledge) and a higher cognitive process.

Now that we know some of the theory, what kind of objectives should we make? Should we focus on medical management? What about crisis-resource management? In part 2, we will tackle what kind of learning objectives should be included in a team-based resuscitation simulated scenario.


References

1) Miller, G. (1990). The assessment of clinical skills/competence/performance. Academic Medicine,65(9), S63-7.

2) Kyle, R. (2008). 7.5 Which of these Curriculum Components are Best Suited to Simulation? In Clinical simulation operations, engineering and management(1st ed., pp. 78-79). Burlington, MA: Academic Press.

3) Bloom, B.S., Engelhart, M.D., Furst, E.J., Hill, W.H., & Krathwohl, D.R. (1956). Taxonomy of educational objectives: The classification of educational goals. Handbook 1: Cognitive domain. New York: David McKay

4) Krathwohl, D. (2002). A Revision Of Bloom’s Taxonomy: An Overview. Theory Into Practice,41(4), 212-218.

Crisis Resource Management

What is CRM?

Crisis Resource Management refers to the extremely important but sometimes difficult to define “soft skills” that can make or break the function of a team. The concept was originally developed by the airline industry in response to research demonstrating that the large majority of airplane crashes occurred due to failures of the crew to effectively utilize resources. In this case, CRM referred to Crew Resource Management, which was a type of training designed to address these issues. Eventually, these ideas were brought to medicine by Gaba, Howard, Fish et al, who developed a curriculum for anesthesiologists.1 This group changed the name of the training to Crisis Resource Management, and the medical field has been calling it crisis resource management ever since.

Being able to identify and label the skill components of CRM helps a simulation educator immensely. It is essential to address these skills during debriefing. In fact, cases can be designed specifically to elicit these skills.

The main components of CRM

Slide1

1) Communication

This should be no surprise. Communication is a basic tenant of good team function in any environment. Classically, good communication during a resuscitation is referred to as “closed loop communication.” This means 1) Clearly identifying who is being spoken to and delegating a clear, specific task to that person. 2) The recipient acknowledging what has been heard. 3) The recipient clearly stating when the delegated task has been completed. This stage is referred to as “closing the loop” on the initial order. Notice that the loop described is for one order. A common communication pitfall is to call out too many orders at once. Quality communication also means listening to suggestions and updates from team members and respectfully acknowledging them.

2) Leadership

There are many ways for the team leader to lead a case. The style of leadership is less important than the fact that there is leadership. Clear communication is part of this. But so is maintaining order and calm in the room, sharing your mental model with the team, and soliciting feedback and ideas from the team. Common pitfalls include not clearly establishing leadership during a resuscitation or having a leader that is not receptive to input from team members.

3) Resource allocation

This refers to the ability to optimize the roles and use of available personnel and equipment. A common pitfall of resource allocation is to forget that there are other resources in the room or outside the room. Does the team leader need to be the person intubating? Does the team leader notice that the medical student is standing in the corner while the nurse doing CPR is getting tired? Would the nurse be of more help administering medications and obtaining IV lines? Do the members of the team ask for help when they need it? Does the team change the monitor to cycle the blood pressure every two minutes instead of every fifteen when the patient status changes from well to unwell?

4) Situational awareness

This refers to the ability of the learner to perceive the many components of their environment. More importantly, it specifically addresses their understanding of what those components mean when combined to one whole. Does the learner recognize that they administered a medication and the patient’s blood pressure immediately dropped? When the patient starts wheezing and the oxygen saturations also drop, does the learner recognize that this could be a consequence of their medication administration? Does she even notice the change in vital signs? Debriefing around situational awareness often involves addressing a failure to recognize a problem, fixation on a single diagnosis or problem, (to the detriment of other possibilities or concurrent problems that require management) or failure to anticipate new problems or complications that may arise as a result of the illness or its treatment.

5) Problem solving

This concept describes the process by which a learner must create a solution to a situation in which there is no routine answer. The process of developing a novel solution can be fraught with cognitive errors. Unpacking these errors can be a very valuable part of debriefing. It is important to note that for very junior learners, almost all situations are unique problems to be solved. (And hence, scenarios often do not need to be particularly complex.) More senior learners require more complex cases simply because they have a broader scope of familiar experiences. In order to challenge their problem solving, one must introduce them to an unexpected complication or a novel patient situation.

References:

These resources all describe the five CRM components listed above. They also look to the assessment of CRM skills.

  1. Gaba DM, Howard SK, Fish KJ, Smith BE, Sowb YA. Simulation-Based Training in Anesthesia Crisis Resource Management (ACRM): A Decade of Experience. Simul Gaming. 2001;32(2):175-193. doi:10.1177/104687810103200206.
  2. Gaba D, Howard S, Fish K. Crisis management in anesthesiology. New York: Churchill Livingstone Publishers; 1994.
  3. Kim J, Neilipovitz D, Cardinal P, Chiu M. A comparison of global rating scale and checklist scores in the validation of an evaluation tool to assess performance in the resuscitation of critically ill patients during simulated emergencies (abbreviated as “CRM simulator study IB”). Simul Healthc. 2009;4:6-16. doi:10.1097/SIH.0b013e3181880472.
  4. Hicks CM, Kiss A, Bandiera GW, Denny CJ. Crisis Resources for Emergency Workers (CREW II): Results of a pilot study and simulation-based crisis resource management course for emergency medicine residents. Can J Emerg Med. 2012;14(Crew Ii):354-362. doi:10.2310/8000.2012.120580.