After a decade in the L&D trenches—from navigating the labyrinth of HIPAA compliance rollouts to building QA checklists that actually survive an audit—I’ve learned one immutable truth: AI is a brilliant intern, but a disastrous subject matter expert. Specifically, when it comes to technical procedures, AI has a bad habit of hallucinating steps that simply do not exist in your organization’s reality.
I keep a "hallucination log" on my desktop. It’s my collection of the most creative, dangerous, and utterly fictional procedural steps generative AI has ever suggested. I’ve seen it invent buttons that don’t exist, add "security approval" workflows that violate company policy, and mandate steps that would effectively brick our production servers. When you are building compliance training, a hallucinated step isn't just a nuisance—it’s a liability.
So, how do we harness the speed of AI without shipping misinformation that puts our learners (and our companies) at risk? It starts with a shift in perspective: move away from the "Looks Good to Me" culture and toward disciplined, risk-based validation.
The Anatomy of a Hallucinated Step
Why does AI add extra steps? It’s not malicious; it’s a predictive machine. If you ask an LLM to "write a process for resetting a user's MFA status," it is looking at millions of general articles about IT help desks. It fills in the gaps with the most probable—not the most accurate—information. If it thinks a step *sounds* logical, it will insert it to make the narrative flow better.
In L&D, we call this a failure of process accuracy. If your employees follow an AI-generated job aid and click a ghost button, they aren't just confused—they are frustrated, and they lose trust in your department. Exactly.. When the stakes are high, a hallucinated step is the difference between a successful deployment and an operational nightmare.
Establishing a Risk-Based Validation Framework
Before you even prompt your AI tool, you need to define the risk. Every piece of content you produce should fall into one of two buckets. I always ask the team: "What is the risk if this is wrong?"
The Risk Assessment Matrix
Risk Level Example Content Validation Strategy Low Stakes Tone-of-voice training, leadership theory, soft skills refresher. Peer review, standard grammar/clarity check. High Stakes Compliance procedures, safety protocols, software updates, InfoSec. Structured SME walkthrough, source-document grounding, QA audit.If you are drafting a high-stakes guide, you cannot treat AI as a primary author. You must treat it as a synthesizer of verified data. If the content falls into the high-stakes category, the "SME Review" is not a suggestion—it is a mandatory gate in your project lifecycle.
Moving Beyond "Looks Good to Me": The Structured SME Walkthrough
I loathe the "Looks Good to Me" sign-off. It’s passive, lazy, and utterly useless during an audit. When an SME says "looks good," they are often skim-reading. To catch hallucinated steps, you need to design a review process that forces engagement.
Instead of sending a PDF to an SME and asking for feedback, try these three techniques:
The "Shadow Walkthrough" Request: Ask your SME to actually perform the process using the content you generated. Tell them: "I need you to follow this step-by-step guide on your test environment. Do not just read it—click the buttons." Negative Affirmation Testing: Specifically ask your SME, "Are there any steps mentioned here that are redundant or do not exist in the current interface?" The "Empty Slot" Method: Provide your SME with a template where they must cite the specific source document (e.g., "Page 4 of the InfoSec Policy v2.1") for every major procedure step. If they can’t link it to a source, it’s a red flag.Fact-Checking and Citation Habits
If you are going to use AI to generate process documentation, you must enforce a "grounding" habit. You cannot simply ask AI to "write a procedure." You must feed the AI your source documents.
The "Grounding" Workflow:
- Source-First Drafting: Paste the actual process documentation into the LLM context window. Use the prompt: "Using only the provided source text, draft a 5-step process guide. If a step is not in the source text, do not invent it. Report any gaps you see." Citation Requirement: Require that every step be tagged with a reference to the source. If the AI cannot generate a citation, the step is likely a hallucination. The Anti-Hallucination Prompt: Always add this constraint: "If you are unsure of a step, state 'I need more information' rather than guessing the process."
Detection and Prevention: Keeping Your Team Honest
How do we ensure this happens? By normalizing the "hallucination log." In our weekly team meetings, we share the weirdest things AI tried to pass off as truth that week. By turning failure into a shared learning experience, you remove the stigma of "AI being wrong" and replace it with a proactive culture of detection.
Strategies for Content Correction
When you discover a hallucinated step during your SME review, don’t just delete it. Treat it as a teaching moment for the project owner. Every piece of content needs a named owner. If we find an error, the owner must be the one to trace it back: Did the prompt create the hallucination? Did we fail to provide the right source documents? Was the SME review process too rushed?
Stop overpromising on AI accuracy. Tell your stakeholders: "AI is providing the first draft, but our team is providing the source-checked, validated reality."
Final Thoughts: Integrity Over Speed
I know the pressure to ship training fast. I know stakeholders want that eLearning module out by EOD. But if you ship a hallucinated step in a compliance document, the time you "saved" in development will be tripled in damage control when an employee violates a policy or crashes a system based on your inaccurate guidance.
We are L&D practitioners. Our job isn't screen reader testing for training content just to produce content; it’s to ensure that the knowledge we transmit is accurate, safe, and actionable. Don't let AI’s predictive nature override your professional judgment. Use your checklists, trust your SMEs, and keep that hallucination log updated. In a world of automated content, the most valuable thing you can offer is the guarantee that your training actually reflects the reality of the work.
Remember: If it’s high-stakes, don't rely on the "Looks Good to Me." Require the walkthrough. Ensure the owner is named. And for heaven's sake, kill the passive voice in your policy documents—if it isn't clear who is doing what, the risk of error increases exponentially.
Have you spotted a wild hallucination in your company's training recently? Drop it in our internal hallucination log or share your process-accuracy win below. I remember a project where learned this lesson the hard way.. Let’s keep each other honest.