Nightmares Case 4: Pulmonary Embolism

This is the fourth in a case series we will be publishing that make up “The Nightmares Course”.

The Nightmares Course at Queen’s University (Kingston, Ontario) was developed in 2011 by Drs. Dan Howes and Mike O’Connor. The course emerged organically in response to requests from first year residents wanting more training in the response to acutely unwell patients. In 2014, Dr. Tim Chaplin took over as the course director and has expanded the course to include first year residents from 14 programs and to provide both formative feedback and summative assessment. The course involves 4 sessions between August and November and a summative OSCE in December. Each session involves 4-5 residents and covers 3 simulated scenarios that are based on common calls to the floor. The course has been adapted for use at the University of Saskatchewan, the University of Manitoba, and the University of Calgary.

Why it Matters

The first few months of residency can be a stressful time with long nights on call and the adjustment to a new level of responsibility. While help should always be available, the first few minutes of managing a decompensating patient is something all junior residents must be competent at. This case series will help to accomplish that through simulation.

Clinical Vignette

It’s 1:00 AM and you’ve been called to assess a 69 year old woman admitted to the Gyne Oncology unit. She was recently diagnosed with ovarian cancer and is actively receiving chemotherapy. Her repeat CT showed decreased tumor burden and the plan is for surgery tomorrow. She was admitted pre-op to receive a blood transfusion for a Hb of 72. The transfusion ended 4 hours ago and was tolerated well. Approximately 30 min ago, the patient started developing shortness of breath and central chest discomfort.

Case Summary

This case involves the approach to the patient with acute dyspnea. The patient is tachypneic but with an otherwise normal respiratory exam. ECG shows new right heart strain. The team should consider multiple possibilities but recognize PE as the most likely cause.

The team is expected to appropriately call for help while initiating management. The patient will decompensate and arrest – thrombolytics should be discussed. After the patient achieves ROSC, the resident will provide handover to the code blue team.

Download here

Nightmare Care #4 – PE

Chest X-ray for the Case

Source: https://openpress.usask.ca/undergradimaging/chapter/pulmonary-thromboembolism/

EKG for the Case

Source: https://litfl.com/ecg-changes-in-pulmonary-embolism/

Nightmares Case 3: Seizure

This is the third in a case series we will be publishing that make up “The Nightmares Course”.

The Nightmares Course at Queen’s University (Kingston, Ontario) was developed in 2011 by Drs. Dan Howes and Mike O’Connor. The course emerged organically in response to requests from first year residents wanting more training in the response to acutely unwell patients. In 2014, Dr. Tim Chaplin took over as the course director and has expanded the course to include first year residents from 14 programs and to provide both formative feedback and summative assessment. The course involves 4 sessions between August and November and a summative OSCE in December. Each session involves 4-5 residents and covers 3 simulated scenarios that are based on common calls to the floor. The course has been adapted for use at the University of Saskatchewan, the University of Manitoba, and the University of Calgary.

Why it Matters

The first few months of residency can be a stressful time with long nights on call and the adjustment to a new level of responsibility. While help should always be available, the first few minutes of managing a decompensating patient is something all junior residents must be competent at. This case series will help to accomplish that through simulation.

Clinical Vignette

It is 01:00 and you are on call covering the thoracic surgery service. You have been called to assess Mr. Wright for a seizure episode.

Case Summary

The resident is called to the ward to manage a patient who may have had a seizure. The patient is somnolent when the resident arrives. Shortly afterward, the patient seizes again. Two doses of anti-epileptic will be required to terminate the seizure. Finally, when the patient has been stabilized, the resident will be required to discuss the case with their staff on call.

Download here

Seizure

Nightmares Case 2: Pneumonia

This is the second in a case series we will be publishing that make up “The Nightmares Course”.

The Nightmares Course at Queen’s University (Kingston, Ontario) was developed in 2011 by Drs. Dan Howes and Mike O’Connor. The course emerged organically in response to requests from first year residents wanting more training in the response to acutely unwell patients. In 2014, Dr. Tim Chaplin took over as the course director and has expanded the course to include first year residents from 14 programs and to provide both formative feedback and summative assessment. The course involves 4 sessions between August and November and a summative OSCE in December. Each session involves 4-5 residents and covers 3 simulated scenarios that are based on common calls to the floor. The course has been adapted for use at the University of Saskatchewan, the University of Manitoba, and the University of Calgary.

Why it Matters

The first few months of residency can be a stressful time with long nights on call and the adjustment to a new level of responsibility. While help should always be available, the first few minutes of managing a decompensating patient is something all junior residents must be competent at. This case series will help to accomplish that through simulation.

Clinical Vignette

Mr. Jim Smith is a 64 year old male that was admitted 3 days ago. He was diagnosed with a community acquired pneumonia and started on daily Moxifloxacin. The nurse is concerned about his increasing shortness of breath since she started the night shift 4 hours ago.

Case Summary

In this case, the patient has been admitted for pneumonia and treated with the usual antibiotics. However, the team has not yet recognized that the causative bacteria is resistant to this antibiotic. The pneumonia has progressed and the team must manage the patient’s respiratory distress and sepsis. The patient requires a change in antibiotics, non-invasive ventilatory support and IV fluid resuscitation.

Download the Case Here

Nightmares Course #2: Pneumonia

EKG for the Case

Pulmonary disease pattern COPD ECG
EKG: https://litfl.com/ecg-in-chronic-obstructive-pulmonary-disease/

Chest X-ray for the Case

Chest X-ray: https://radiopaedia.org/cases/right-upper-lobe-pneumonia-8

Nightmares Case 1: Bradycardia

This is the first in a case series we will be publishing that make up “The Nightmares Course”.

The Nightmares Course at Queen’s University (Kingston, Ontario) was developed in 2011 by Drs. Dan Howes and Mike O’Connor. The course emerged organically in response to requests from first year residents wanting more training in the response to acutely unwell patients. In 2014, Dr. Tim Chaplin took over as the course director and has expanded the course to include first year residents from 14 programs and to provide both formative feedback and summative assessment. The course involves 4 sessions between August and November and a summative OSCE in December. Each session involves 4-5 residents and covers 3 simulated scenarios that are based on common calls to the floor. The course has been adapted for use at the University of Saskatchewan, the University of Manitoba, and the University of Calgary.

Why It Matters

The first few months of residency can be a stressful time with long nights on call and the adjustment to a new level of responsibility. While help should always be available, the first few minutes of managing a decompensating patient is something all junior residents must be competent at. This case series will help to accomplish that through simulation.

Clinical Vignette

The triage note states – Patient “fainted” while returning from the bathroom at home. He was found to be slightly more confused by his wife and complained of right elbow pain.

Case Summary

This is a case of an elderly patient with syncope. He is found to be in third degree heart block.  The team is expected to perform an initial assessment and obtain an ECG. Upon recognizing the heart block, they should ensure IV access and place pacer pads while calling for help.

Download the case here:

Bradycardia

ECG for the case found here:

Brady.jpg

Source: https://www.ecgquest.net/ecg/complete-heart-block-3/

Geriatric Case 6: Elder Abuse

This is the sixth and final case in a six-part mini-series focusing on the management of geriatric patients in the ED. This series of cases was written by Drs. Rebecca Shaw, Nemat Alsaba, and Victoria Brazil.

Dr. Rebecca Shaw is an emergency physician currently working as a medical education fellow within the Emergency Department of the Gold Coast Hospital and Health Service in Queensland, Australia. Dr. Nemat Alsaba (@talk2nemat) is an emergency physician with a special interest in geriatric emergency medicine, medical education and simulation. She is trying her best to combine these interests to improve geriatric patient care across all health sectors. She is also an assistant professor in medical education and simulation at Bond university. Dr. Victoria Brazil is an emergency physician and medical educator. She is Professor of Emergency Medicine and Director of Simulation at the Gold Coast Health Service, and at Bond University medical program. Victoria’s main interests are in connecting education with patient care – through healthcare simulation, technology enabled learning, faculty development activities, and talking at conferences. Victoria is an enthusiast in the social media and #FOAMed world (@SocraticEM), and she is co-producer of Simulcast (Simulationpodcast.com).

Why It Matters

Elder abuse and neglect is under-recognized, under-reported and under-treated. The emergency department provides an opportunity to identify and intervene in cases of elder abuse. Often, the signs of abuse may be subtle. This case gives participants the chance to improving their skill in identifying elder abuse and to practice their approach to this emotionally challenging issue.

Clinical Vignette

A bedside RN comes to you and says, “Nora has been brought into ED after a fall at home 3 days ago. She is a bit tachycardic and complaining of some pain in her abdomen. She has a few bruises on the rest of her body. Could you please assess her?”

Case Summary

An 80-year old woman presents after a fall at home. She is complaining of right sided upper abdominal pain since the fall. She is also complaining of intermittent palpitations and dizziness prior to the fall. Participants are expected to identify that the cause of the fall is due to elder abuse and to manage this along with her concurrent medical issues and abdominal injury.

Key to a Successful Simulation

This case uses a standardized patient who has an extensive script and back story. This patient needs to be familiar with the story and respond in character to the participants questions and empathy (or lack of empathy). There should be a slow unfolding of the story as the participants gain the patient’s trust.

Download the case here:

ECG for the case:

ECG Source: Dr Ed Burns, LITFL.com

Chest x-ray for the case:

Pelvis x-ray for the case:

RUQ ultrasound for the case:

U/S source: McMaster PoCUS Subspecialty Training Program

Geriatric Case 5: Trauma with Head Injury

This case is the fifth in a six-part mini-series focusing on the management of geriatric patients in the ED. This series of cases was written by Drs. Rebecca Shaw, Nemat Alsaba, and Victoria Brazil.

Dr. Rebecca Shaw is an emergency physician currently working as a medical education fellow within the Emergency Department of the Gold Coast Hospital and Health Service in Queensland, Australia. Dr. Nemat Alsaba (@talk2nemat) is an emergency physician with a special interest in geriatric emergency medicine, medical education and simulation. She is trying her best to combine these interests to improve geriatric patient care across all health sectors. She is also an assistant professor in medical education and simulation at Bond university. Dr. Victoria Brazil is an emergency physician and medical educator. She is Professor of Emergency Medicine and Director of Simulation at the Gold Coast Health Service, and at Bond University medical program. Victoria’s main interests are in connecting education with patient care – through healthcare simulation, technology enabled learning, faculty development activities, and talking at conferences. Victoria is an enthusiast in the social media and #FOAMed world (@SocraticEM), and she is co-producer of Simulcast (Simulationpodcast.com).

Why It Matters

Elderly patients who have sustained trauma are frequently encountered in the ED. These patients have unique physiology and are often complex due to frailty and polypharmacy concerns. Care of the elderly trauma patient requires attention to these complexities, to goals of care, and to communication with family members. This case gives the opportunity to learn and enhance these skills.

Clinical Vignette

The bedside nurse informs you that “EMS just off-loaded an elderly male to the resuscitation bay. He had a fall down the stairs and sustained a head injury. He was GCS 15 and hemodynamically stable when they picked him up, so they didn’t activate the trauma team, but he has deteriorated during transport. He has an obvious large, boggy scalp hematoma over the left parietal region. I am worried because he’s getting restless and won’t follow commands.”

Case Summary

An 81-year old man falls down the stairs at home. He is initially asymptomatic but his level of consciousness declines and he starts to show signs of raised ICP. Providers must recognize and treat this, as well as reverse his anticoagulation, provide neuroprotective RSI and safely transport to the CT scanner. Providers must then talk with the patient’s wife, to provide information on his condition and prognosis and discuss the patient’s goals of care.

Download the case here:

Geriatric Trauma with Head Injury

ECG for the case found here:

Geriatric Trauma ECG

ECG Source: https://en.ecgpedia.org/index.php?title=Atrial_Fibrillation

CXR for the case found here:

Geriatric Trauma CXR

Image courtesy of Dr Jeremy Jones, Radiopaedia.org, rID: 6410

Pelvic XR for the case found here:

Geriatric Trauma Pelvic XR

Image courtesy of Dr Jeremy Jones, Radiopaedia.org, rID: 28928

Geriatric Case 4: End of Life Care

This case is the fourth in a six-part mini-series focusing on the management of geriatric patients in the ED. This series of cases was written by Drs. Rebecca Shaw, Nemat Alsaba, and Victoria Brazil.

Dr. Rebecca Shaw is an emergency physician currently working as a Medical Education Fellow within the Emergency department of the Gold Coast Hospital and Health Service in Queensland, Australia. Dr. Nemat Alsaba (@talk2nemat) is an Emergency physician with a special interest in Geriatric Emergency Medicine, medical education and simulation. She is trying her best to combine these interests to improve Geriatric patient care across all health sectors. She is also an Assistant professor in medical education and simulation at Bond university. Dr. Victoria Brazil is an emergency physician and medical educator. She is Professor of Emergency Medicine and Director of Simulation at the Gold Coast Health Service, and at Bond University medical program. Victoria’s main interests are in connecting education with patient care – through healthcare simulation, technology enabled learning, faculty development activities, and talking at conferences. Victoria is an enthusiast in the social media and #FOAMed world (@SocraticEM), and she is co-producer of Simulcast (Simulationpodcast.com).

Why it Matters

Elderly patients requiring resuscitation are frequently encountered in the ED. When patients are non-communicative, close family members are regularly required to act as substitute decision makers and represent their family member’s wishes. Engaging and communicating effectively with SDMs in end-of-life and goals-of-care discussions is necessary to provide the most appropriate care for the elderly patient. This case gives the opportunity to learn and enhance these skills.

Clinical Vignette

The charge nurse informs you “I just put a very unwell looking patient into resus. She’s from a nursing home facility and the paramedics think she is septic. She’s hypotensive and barely responsive. Honestly, she looks like she might be dying. Her granddaughter is on her way. I don’t think she has a known advanced care directive or code status.”

Case Summary

An 89-year-old patient is brought in to the ED by ambulance from their nursing home. Staff found her unresponsive and hypotensive at morning handover. She had been treated for UTI by her family physician over the last few days. Participants identify severe sepsis and realize that critical care interventions may be inappropriate. This should prompt a goals of care discussion including potential for initiating end-of-life care.

Download the case here:

ECG for the case found here:

ECG source: https://litfl.com/hyperkalaemia-ecg-library/

CXR for the case found here:

CXR source: https://emrems.com/2013/01/30/how-to-you-tell-its-a-right-middle-lobe-infiltrate/

Getting Serious about GridlockED: Lesson Plans to Teach about Systems Improvement

Written by Sonja Wakeling. Edited by Dr. Teresa Chan.

Everything I know about ED management I learned from… A Board Game?

GridlockED is an innovative board game that fosters teamwork, knowledge acquisition and application, and problem-solving skills. Developed by clinician educators and trainees, it was designed to simulate real-life settings in an emergency department within a risk-free learning environment. In the healthcare field, it is impossible to allow junior learners full reign of an emergency department, yet they require some level of experience if they are to be responsible and effective when they are practicing and learning. It is imperative that learners exercise and develop skills in a variety of required domains, such as the Royal College of Physicians and Surgeons of Canada’s CanMEDs qualities; these include communication, collaboration, health leadership, health advocacy, scholarship, and professionalism..(1)

Learners acquire knowledge in a variety of approaches, traditionally through didactic lectures but also through other more interactive methods. There has been an important shift from classic knowledge dissemination to more active participation(2); however, finding novel ways to provide both effective and efficient acquisition assists in training a highly-qualified generation of new physicians. In recent years, there has been a surge of simulation-based learning in medical education, particularly at the level of post-graduate and undergraduate medical training.(3) Multiple studies have shown that simulations are an effective method of education; for example, increasing the learner’s confidence in addition to increasing knowledge retention both short- and long-term.(4)

GridlockED: a serious game for learners

Serious games, which Bergeron defines as an “interactive computer application, with or without significant hardware components”, are a form of simulations.(5) Instead of being designed with a set of primary winning objectives, their main objective is for the player to acquire knowledge and skills in a challenging and fun learning environment; in a situation like this, the knowledge is seemingly acquired with little effort.(6) This method of learning has recently taken off in residency education, as evidenced by the systematic review of serious gaming within the surgical field(6); however, there is limited evidence to support their utility given that the expanding use of serious gaming as a relatively new approach to medical education. This is one such area that require intensive and thorough research as a means of advancing effective teaching methods in medical education.

In the context of disaster preparedness, tabletop exercises and simulations have been a key aspect of their planning and preparations.(7) GridlockED is an example of a serious game that employs a tabletop simulation approach, which Agboola an colleagues described as one that “involves key personnel discussing simulated scenarios in an informal setting based on existing operational plans and identifying where those plans need to be refined.”(7) GridlockED can accommodate up to 6 participants, given that it is a co-operative and collaborative style of play, whose purpose is to collaborate and reason through the management of patients during a “standard” 8-hour shift in an emergency department. For those with ample professional experience, this task certainly may not seem difficult; however, as a junior learner it sets a great challenge to efficiently and intelligently manage the flow of patients.

Objectives and templates: GridlockED as a teaching tool

So, what makes GridlockED a valuable teaching tool? This low-stakes learning environment allows participants the opportunity to discuss various approaches, make mistakes (and more importantly, to learn from them), and come to understand how they might prioritize certain patients or tasks in a busy emergency department. The point is not to acquire knowledge around diseases or illness management, but rather the skill to lead and collaborate in a mission to provide effective care (and ultimately “win” the game). The beauty of this game is that this is all done outside of the department where there are no real patients, and where entrusted facilitators can help provoke discussion around challenges, successes, and errors.

In addition to the basic gameplay set-up, learning templates are being developed that focus on themed settings or situations that promote particular clinical lessons. For example, there is one theme that results in a shortage of nursing staff, forcing participants to adjust their gameplay style and therefore clinical management akin to a similar situation in a real emergency department. Another theme places you in a rural emergency department with limited resources including staff and specialists available to assist you. Furthermore, an additional template fills your department with patients, taking the game title GridlockED to a serious level. Each teaching template is led by a facilitator who helps the team delve into the decisions they make and reflect on methods to improve their future management.


Case 1 – Best Shift Ever (218 kb)

Case 2 – Rural Hospital (226 kb)

Case 3 – Where have all the nurses gone? (216 kb)

Case 4 – Safety Worries (202kb)

Case 5 – Overwhelming Diagnostic Imaging (222 kb)

Case 6 – The Critical Consultant (182 kb)

Case 7 – Night(mare) Shift (234kb)

Case 8 – Code Gridlock! (210 kb)

Download all cases at once. (1.6 MB)


Lessons learned, future patients saved

As a junior learner myself, I have taken some key learning points away from each round of GridlockED I have participated in; for example, never forget the bigger picture. The game allows you to slow down for a moment and view the whole “picture” of the department. Here, you can keep a watchful eye on both patient flow and volume, consider the challenges you are currently facing, and what you foresee may occur. Layer on top of this basic gameplay with various themed learning templates and you have yourself a robust and effective teaching tool that is also fun to engage in!

But there are many more valuable learning points I have taken away from each cycle of gameplay lend themselves to the management of a real emergency department.

  1. Plan ahead. It is not a good idea to leave your high acuity beds with unstable patients in them, unless you have absolutely no choice. If you cannot care for the next patient who comes in in serious condition because your beds are blocked, you (and that patient) are in trouble. And related to that, strategize to maximize the efficiency in each zone of the department.
  2. Prioritize sick patients. Despite patients expressing concern about wait times, there are instances where it is reasonable and indeed appropriate to delay care of low-acuity patients in favour of those who are in serious condition. If someone is waiting in an emergency department, and given the limitations of the healthcare system, it is often a good sign; that individual is not dying or in critical condition, so it means there are other patients who require more immediate attention. That is not a position anyone wants to be in, so try to wait with patience and gratitude.
  3. Collaborate inter-professionally. There are various roles to be played, including nursing, specialties, and learners. However, each participant playing may have a different role in the real world. Rely on their knowledge and experience, and listen to what they say. The most effective leaders know when to lead and also when to listen.

Speaking of collaboration, do not forget the limitations of each allied health or specialty role. Recognize when a professional may be in over their head and help where you can. Only so many blood vials or imaging results can be completed in a fixed amount of time. Be patient, be mindful, and have reasonable expectations.

At the end of the day, do what is best for the patients being treated. See as many patients as you can, treat as many as you can, and do your best to save everyone you can. Sometimes you cannot save everyone, but learn from each experience you have so you improve the situation for the next time you have a similar scenario.

Now that you know what is at stake… Are you up for the challenge?


GridlockED was developed by staff physicians and medical students at McMaster University in Hamilton, Ontario, Canada. Please visit https://gridlockedgame.com if you would like to learn more or purchase the game. All proceeds for the game go towards fostering further education and scholarly projects at McMaster University.


References

  1. Royal College of Physicians and Surgeons of Canada: CanMEDS Framework. http://www.royalcollege.ca/rcsite/canmeds/canmeds-framework-e. Accessed September 3, 2018.
  2. Allerly LA. Educational games and structured experiences. Med Teach. 2004 Sep;26(6):504-5.
  3. Bradley P. The history of simulation in medical education and possible future directions. Med Educ. 2006 Mar;40(3):254-62.
  4. Behar S, Upperman JS, Ramirez M, Dorey F, Nager A. Training medical staff for pediatric disaster victims: a comparison of different teaching methods. Am J Disaster Med. 2008 Jul-Aug;3(4):189-99.
  5. Bergeron BP. Developing Serious Games. Charles River Media: Hingham, 2006.
  6. Graafland M, Schraagen JM, Schijven MP. Systematic review of serious games for medical education and surgical skills training. Br J Surg. 2012;99:1322-1330.
  7. Agboola F, McCarthy T, Biddinger PD. Impact of emergency preparedness exercise on performance. J Public Health Manag Pract. 2013 Sep-Oct;19 Suppl 2:S77-83.

Limiting Gender Bias in Simulation Assessment

Today’s piece is written by Dr. Lall. She is an Associate Professor and Associate Residency Director of Emergency Medicine at Emory University in Atlanta, GA. She is also the current president of the Academy for Women in Academic Emergency Medicine. Dr. Lall’s research focuses include physician wellness and gender bias and inequity in medicine. The following is a summary of her recent publication on this issue.

You can find the publication here:

Jeffrey N. SiegelmanMichelle LallLindsay LeeTim P. MoranJoshua Wallenstein, and Bijal Shah (2018) Gender Bias in Simulation-Based Assessments of Emergency Medicine Residents. Journal of Graduate Medical Education: August 2018, Vol. 10, No. 4, pp. 411-415.

Background:

There is a paucity of studies on gender differences in milestone assessment. One recent large multi-site cohort study of EM residents evaluated bias in end-of shift evaluations and found a significant bias based on resident gender (Dayal A et al, 2017).  Shift evaluations usually represent subjective assessments and residents are evaluated only on cases seen during a particular shift, resulting in considerable variation with respect to which competencies are assessed across residents and rated by faculty. Simulation allows for a more structured, consistent evaluation environment in which residents can be tested on identical clinical problems, and in which specific competencies can be assessed. We hypothesized that simulation, being a more objective assessment tool, may mitigate gender disparities in resident assessment.

In our three year experience with biannual milestone-based simulation assessments of all our EM residents, no significant gender bias was observed in contrast to other forms of resident assessments, such as end-of-shift evaluations.

Tips for SIM Educators:

  1. Training the standardized patient is key to successful simulation assessment.
    1. Pilot test the scenarios to ensure the case plays as expected and appropriately elicits the opportunity for the resident to perform the desired critical behaviors.
    2. Evaluate for potential bias introduced by the standardized patient script or actions as the scenario plays out.
    3. Ensure that standardized patient responses are the same every time.
      1. Same response in the same tone of voice with the same facial expressions whether the physician is male or female.
    4. Standardized patient script cues should be written with binary language.
      1. If the resident does not introduce themself to the patient, prompt the resident with “Doctor, what is your name?”
      2. Avoid language like miss, ma’am or sir
  2. Education and training of the rater is of critical importance.
    1. Raters should be instructed that evaluation in these cases is not subjective.  Evaluation is binary and based on observable behavior only.
  3. Convert milestone language into binary, observable behaviors
    1. Assessment items should avoid language that may introduce bias including subjective assessments.
      1. Agenic adjectives: typically used to describe men and when used to describe women carry a negative connotation.  Examples include assertive, autonomous, independent, confident, intellectual.
      2. Communal adjectives: typically used to describe women and when not demonstrated by women carry a negative connotation.  Examples include kind, compassionate, sympathetic, warm, helpful.
    2. Focus on action based assessment items, for example:
      1. Resident introduced themself to the patient
      2. Resident updated the family using lay terminology
      3. Resident ordered magnesium without prompting

Sim Checklist 3

Checklist for Limiting Bias in Simulation Assessment

  • Standardize the Scenario
    • Standardized patient/Confederate scripting and training is crucial
    • Simulation operator training
    • Pilot the case
  • Create an Objective Rating Tool
    • Focus on observable behaviors rather than subjective assessments
      • Observed/ Not Observed/ Unable to Assess
    • Train the raters
    • Use language that avoids bias
  • Monitor for bias
    • Analyze data after an assessment for validity evidence, reliability, and evidence of bias
    • Make adjustments to the case as needed