In many ways, Cognitive Arts is defined by our innovative approach to designing and deploying learning programs. Using our proprietary methodology – Critical Mistake Analysis (CMA) – we help organizations and training departments take a fact-based approach in deciding what to teach. And once those decisions are made, our approach translates the most important concepts into goal-based scenarios, conveyed through interactive learning environments using games and simulations. This creates opportunities for learners to practice targeted skills in a realistic but safe environment that recreates the conditions that lead to common mistakes, experience the consequences of those mistakes, and receive coaching on how to do better.
CMA methodology uses empirical data to identify the most important issues that a course should focus on. By identifying the most common and costly mistakes made by novices deploying target skills in the real world, and performing a root-cause analysis to determine the underlying causes of these mistakes, Critical Mistake Analysis tells you exactly what content will have the most significant business impact.
CMA methodology evolved from extensive research conducted at Northwestern University's Institute for the Learning Sciences. NIIT and Cognitive Arts have proven the value of this approach in numerous client engagements across a variety of industries and content areas.
Critical mistake analysis is a five-step process:
Step 1: Identify the critical mistakes
The first step is to identify the mistakes that novices executing targeted skills make. This information is gathered through a combination of surveys, interviews, and on-the-job observations, combined with whatever operational data is available.
Step 2: Identify the most common mistakes
Having identified the mistakes that are being made in the field, our instructional designers then do further analysis to determine the frequency and impact of each mistake. These quantities multiplied together give an estimate of the overall impact of this type of mistake on the organization. The distribution of impacts roughly obeys a Pareto, or "80/20" distribution, which means that the top 20 percent of observed mistakes will typically account for 80 percent of the overall impact of all mistakes. This is captured in a classic Pareto chart, like the one below, which documents the relative impact of mistakes observed for a retail client.
The Pareto chart method is used to prioritize observed mistakes to facilitate the selection of the ones that will receive priority in the training content to be developed - what we call the "critical mistakes."
Step 3: Perform a root cause analysis of the mistake
Once CMA has identified the critical mistakes, the next step is to perform a root cause analysis on each mistake to determine why learners make it. This analysis focuses on three key issues: first, the decision that leads to the mistake; second, the contextual factors that make the mistake likely; and third, the underlying beliefs that lead people to make the wrong decision.
Step 4: Create "teaching points"
The output of a Critical Mistake Analysis is encoded in a set of "teaching points", that encapsulate our understanding of each mistake in a way that facilitates the development of training to remediate it.
Teaching points encompass the material from the root cause analysis combined with the correct alternative to the mistake, and the underlying reasons why this is preferable. In other words, a teaching point encodes a description of a mistake, an analysis of why that mistake is made, and an explanation of how to avoid it and why that is preferable.
Step 5: Build from the teaching points
Once a set of teaching points has been developed based on a critical mistake analysis, the final step is to create training that addresses each of these points effectively. The fundamental approach to this is the goal-based scenario. Goal-based scenarios make it possible for learners to practice skills and acquire relevant experience without the potentially negative consequences of making mistakes in real life. When appropriately designed, such scenarios provide an engaging experience for the student, and can be delivered in a variety of ways.
In a goal-based scenario, the learner is given a mission, or goal, and an opportunity to practice trying to achieve that goal in a realistic environment. As the learner pursues the goal, he or she faces challenges that are based on the teaching points identified through a critical mistake analysis. When the learner makes one of the critical mistakes, he or she is supported with focused coaching designed to help in analyzing the reasons for the mistake, and ultimately, in learning from the experience.
CMA methodology, which has been applied to more than 100 programs for dozens of corporations worldwide, has proven effective for individual learners as well as businesses working to align human performance with organizational goals. To learn more, please download this white paper about CMA.
