NDM and COVID-19

The COVID-19 pandemic fits the Naturalistic Decision Making (NDM) profile: vague goals, multiple players, time pressure, uncertainty, importance of expertise, organizational constraints, high stakes, dynamic settings.

Features of Naturalistic Decision Making

Source: Gary Klein

But what tools does the NDM community have to offer? I have gathered a range of possibilities and then down-selected to highlight eight tools that might be useful for our current crisis.

Expertise and discoveries. NDM has several tools for capturing and transferring expertise.

• Cognitive Task Analysis (CTA) methods. Expertise needs to be transferred (e.g., expertise in using ventilators), and expertise primarily hinges on tacit knowledge (Crandall, Klein, & Hoffman, 2006). CTA methods can be invaluable for eliciting expertise. It is also possible to train healthcare workers to do their own CTA in order to harvest their own knowledge. In the murky world of COVID-19, front-line healthcare workers are making discoveries and it seems likely that many important discoveries aren’t captured or shared.
• Assessing expertise. Who is an expert? This question is always hard and has become harder because of the novelties involved in COVID-19. The measures we used to identify experts in the past may no longer be relevant today. We have identified seven criteria for identifying experts — and described the limitations of each of those criteria.
• ShadowBox. All kinds of training requirements are emerging but how can organizations conduct training during social distancing? ShadowBox is designed for distance learning, and for training cognitive skills. The Snapshot version of ShadowBox can address mindset shifts, and the Cue Detect version can address perceptual skills.
• Evidence-Based Medicine (EBM). As much as we would like to rely on scientific evidence for managing and treating COVID-19, we can’t wait several years to design rigorous studies. Worse, some of the practices of the past may be counterproductive in handling COVID-19. Plus, there is a value now of identifying positive deviants — people who defy conventional wisdom and have learned to be more effective as a result. In other words, EBM may not be working very well. D. Klein and colleagues (2016) have described the cognitive challenges of relying on EBM — they want to prevent organizations from getting hamstrung by EBM.

Replanning. In a coherent, predictable world our leaders would carefully design plans to be carefully carried out. However, we are not living in that world. We will need to try to put plans into action but we have to expect that the plans will be modified — the goals may have to be revised or possibly discarded and replaced, based on what we learn.

• Flexecution/Management by Discovery. In this fluid environment, we are dealing with wicked problems. Management by Discovery offers a counterpoint to objectives-based planning.
• Pre-mortem. We don’t have clear evaluation criteria for assessing plans in the kind of fluid and complex challenges we are facing. Under these conditions, the Pre-mortem method can be useful for flagging potential problems with plans we want to put into action, and/or for anticipating the consequences of a decision not to act.

Communicating. There is no shortage of methods to support communication. But here are two methods that have emerged from NDM, one to communicate advanced models and technologies and the other to reduce cross-cultural confusions.

• Cognitive Tutorial. Public officials are trying to make critical decisions based on various models, but they admittedly do not understand these models, don’t understand how they might apply to their locale, and do not understand the limitations and weaknesses of the models. The Cognitive Tutorial that Mueller & Klein developed almost a decade ago (Mueller & Klein, 2011) can be quite valuable to helping decisionmakers achieve better use of models and other sophisticated tools.
• Cultural Lens model. Given the requirement for international cooperation, one barrier is that people from different national groups don’t line up in the way they think — the kinds of arguments and evidence they respond to. The cultural lens model (e.g., H. A. Klein and Kuperman, 2008) is designed to reduce these kinds of cultural confusions.

The NDM community has additional methods and tools, but the eight listed in this essay seem like a good start for helping decision makers, individually and organizationally, wrestle with the ambiguities and complexities we are facing.