GenAI outputs are inherently probabilistic and subject to errors ranging from hallucinations and bias to contextual misalignments.
Unlike traditional systems, GenAI error states are hard to predict. Designing for these states requires transparency, recovery mechanisms and user agency. A well-designed error state can help users understand AI system boundaries and regain control.
A Confusion matrix helps analyse AI system errors and provides insight into how well the model is performing by showing the counts of
True positives (correctly identifying a positive case)
False positives (incorrectly identifying a positive case)
True negatives (correctly identifying a negative case)
False negatives (failing to identify a negative case)
Scenarios of AI errors and failure states
System failure (wrong output)
False positives or false negatives occur due to poor data, biases or model hallucinations. E.g., Citibank financial fraud system displays a message “Unusual transaction. Your card is blocked. If it was you, please verify your identity”
System limitation errors (no output)
True negatives occur due to untrained use cases or gaps in knowledge. E.g., when an ODQA system is given a user input outside the trained dataset, throws the following error “Sorry, we don’t have enough information. Please try a different query!”
Contextual errors (misunderstood output)
True positives that confuse users due to poor explanations or conflicts with user expectations comes under contextual errors. E.g., when user logs in from a new device, gets locked out. AI responds: “Your login attempt was flagged for suspicious activity”
How to use this pattern
Communicate AI errors for various scenarios: Use phrases like “This may not be accurate”, “This seems like…” or surface confidence levels to help calibrate trust.
Use pattern convey model confidence for low confidence outputs.
Offer error recovery: Incase of System failure or Contextual errors, provide clear paths to override, retry or escalate the issue. E.g., Use way forwards like “Try a different query,” or “Let me refine that.” or “Contact Support”.
Enable user feedback: Make it easy to report hallucinations or incorrect outputs. Read more about pattern 19. Design to capture user feedback.