Hallucination in AI refers to the agent fabricating information that is not true or not supported by its data. In many regulated environments, providing incorrect information is a serious issue. Even in general customer service, an AI making up a policy or a detail can erode trust. We need to benchmark how often (if ever) the agent produces factually incorrect or nonsensical statements. This can be done by having a set of factual questions or scenarios where the correct answer is known and seeing if the agent’s responses match the truth. If using a generative model, one might incorporate an automated fact-checking step or an LLM-as-a-judge approach to flag likely hallucinations. For instance, ask the agent a question with no answer in its knowledge base (to tempt a hallucination) – does it gracefully say it doesn’t know, or does it make something up? Track the hallucination rate (perhaps percentage of answers that contain false information in test queries). The goal is to minimize this. In critical areas, ideally the agent should have 0 unauthorized improvisations. If any hallucination is detected in testing, that’s a bug to fix (by refining prompts, adding fallback rules, etc.). Auditing conversations for accuracy is also important in production; for testing, one might use a smaller set of queries with known answers to regularly sanity-check the AI’s responses.