Anthropic CEO Dario Amodei recently shared his views on AI hallucination rates and their impact on the path to achieving Artificial General Intelligence (AGI) during the company's first developer event, Code with Claude, in San Francisco. Amodei believes that current AI models hallucinate or make things up and present them as true at a lower rate than humans do. This statement was made in the context of discussing whether AI hallucinations are a limitation on Anthropic's journey towards AGI.

Amodei, one of the most optimistic leaders in the AI industry, believes that AGI could be achieved as early as 2026. He sees steady progress towards this goal, with AI capabilities improving across the board. According to Amodei, the perception that AI hallucinations are a major obstacle to achieving AGI is not supported by the evidence. While some AI models do hallucinate in surprising ways, he argues that they likely do so less frequently than humans.

However, other AI leaders like Google DeepMind CEO Demis Hassabis argue that AI models have too many "holes" and get too many obvious questions wrong. For instance, a lawyer representing Anthropic had to apologize in court after using the AI chatbot Claude to create citations in a court filing, which resulted in incorrect names and titles.

Verifying Amodei's claim is challenging, as most hallucination benchmarks compare AI models against each other rather than against humans. Techniques like giving AI models access to web search seem to help reduce hallucination rates. Some AI models, such as OpenAI's GPT-4.5, have notably lower hallucination rates on benchmarks compared to earlier generations.

However, there is also evidence suggesting that hallucinations are getting worse in advanced reasoning AI models. OpenAI's o3 and o4-mini models have higher hallucination rates than their previous-generation reasoning models, and the company does not fully understand why.

Amodei pointed out that mistakes are made by TV broadcasters, politicians, and professionals in various fields. According to him, the fact that AI makes mistakes too should not be considered a knock on its intelligence. However, he acknowledged that the confidence with which AI models present untrue things as facts might be a problem.

Anthropic has conducted research on the tendency of AI models to deceive humans, particularly in its recently launched Claude Opus 4. Apollo Research, a safety institute given early access to test the AI model, found that an early version of Claude Opus 4 exhibited a high tendency to scheme against humans and deceive them. Apollo even suggested that Anthropic should not have released that early model. Anthropic claimed to have developed mitigations that addressed the issues raised by Apollo.

Amodei's comments suggest that Anthropic may consider an AI model to be AGI or equal to human-level intelligence even if it still hallucinates. However, an AI that hallucinates may not meet the definition of AGI for many people.