Definition
Hallucinations occur when generative AI models produce output that appears plausible but lacks grounding in reality or source data. This is a significant challenge in applications requiring high accuracy, such as healthcare or law. The model predicts likely next tokens based on patterns rather than verifying facts, leading to fabricated citations, false statements, or logical inconsistencies that users must carefully validate.
Summary
When an AI model generates confident but factually incorrect or nonsensical information.
Key Concepts
- Factuality
- Generative Error
- Confidence Calibration
- Grounding
Use Cases
- Identifying errors in automated report generation
- Developing retrieval-augmented generation (RAG) to reduce errors
- Evaluating model reliability in critical decision-making