In the fiercely competitive landscape of artificial intelligence, Apple has recently announced updates to its AI models, which power various Apple Intelligence features across iOS, macOS, and other platforms. However, the company's own benchmarks reveal that these models underperform compared to older models from rival tech firms, including OpenAI.

Apple On-Device: A Mixed Reception

Apple's latest "Apple On-Device" model, designed to run offline on products like the iPhone, has been rated by human testers as generating text of comparable quality to similarly sized models from Google and Alibaba. However, it does not outperform these rivals. With approximately 3 billion parameters, this model drives features such as summarization and text analysis.

Apple Server: Failing to Impress

The more capable "Apple Server" model, designed to run in Apple's data centers, has also received mixed reviews. Human raters preferred Meta's Llama 4 Scout model over Apple Server in tests evaluating the ability to analyze images. This is surprising, as Llama 4 Scout has been outperformed by leading models from AI labs like Google, Anthropic, and OpenAI in numerous tests.

Benchmark Results and AI Struggles

These benchmark results lend credibility to reports suggesting that Apple's AI research division has struggled to keep pace with competitors in the AI race. Apple's AI capabilities in recent years have been underwhelming, and a promised Siri upgrade has been delayed indefinitely. Some customers have even sued Apple, accusing the company of marketing AI features for its products that it has not yet delivered.

Improved Tool Use and Efficiency

Despite these setbacks, Apple claims that both Apple On-Device and Apple Server boast improved tool use and efficiency compared to their predecessors. They can understand around 15 languages, thanks in part to an expanded training dataset that includes image data, PDFs, documents, manuscripts, infographics, tables, and charts.

Third-Party Developers Gain Access

As of Monday, third-party developers can tap into the Apple On-Device model via Apple's Foundation Models framework. This move could potentially accelerate the adoption and development of AI-powered features and applications.

In conclusion, while Apple's AI models show some progress, they have yet to surpass the performance of their competitors. The company's AI research division faces an uphill battle in the cutthroat AI race, as it seeks to deliver on the promises of advanced AI capabilities in its products.