Leading artificial intelligence companies including OpenAI, Google, and Anthropic are facing "diminishing returns" from their costly efforts to build newer AI models, according to a new Bloomberg report. The stumbling blocks appear to be growing in size as Apple continues a phased rollout of its own AI features through Apple Intelligence.
OpenAI's latest model, known internally as Orion, has reportedly fallen short of the company's performance expectations, particularly in handling coding tasks. The model is said to be lacking the significant improvements over existing systems when compared to the gains GPT-4 made versus its predecessor.
Google is also reportedly facing similar obstacles with its upcoming Gemini software, while Anthropic has delayed the release of its anticipated Claude 3.5 Opus model. Industry experts who spoke to Bloomberg attributed the challenges to the increasing difficulty in finding "new, untapped sources of high-quality, human-made training data" and the enormous costs associated with developing and operating new models concurrently with existing ones.
Silicon Valley's belief that more computing power, data, and larger models will inevitably lead to better performance, and ultimately the holy grail – artificial general intelligence (AGI) – could be based on false assumptions, suggests the report. Consequently, companies are now exploring alternative approaches, including further post-training (incorporating human feedback to improve responses and refining the tone) and developing AI tools called agents that can perform targeted tasks, such as booking flights or sending emails on a user's behalf.
"The AGI bubble is bursting a little bit," said Margaret Mitchell, chief ethics scientist at AI startup Hugging Face. She told Bloomberg that "different training approaches" may be needed to make AI models work really well on a variety of tasks. Other experts who spoke to the outlet echoed Mitchell's sentiment.
How much impact these challenges will have on Apple's approach is unclear, though Apple Intelligence is more focused in comparison, and the company uses internal large language models (LLMs) grounded in privacy. Apple's AI services mainly operate on-device, while the company's Private Cloud Compute encrypted servers are only pinged for tasks requiring more advanced processing power.
Apple is integrating AI capabilities into existing products and services, including writing tools, Siri improvements, and image generation features, so it can't be said to be competing directly in the LLM space. However, Apple has agreed a partnership with OpenAI that allows Siri to optionally hand off more open-ended queries to ChatGPT. Apple has also reportedly held discussions with other LLM companies about similar outsourcing partnerships.
It's possible that the challenges faced by major AI companies pursuing breakthrough general-purpose AI models could ultimately validate Apple's more conservative strategy of developing specific AI features that enhance the user experience. In that sense, its privacy-first policy may not be the straitjacket it first seemed. Apple plans to expand Apple Intelligence features next month with the release of iOS 18.2 and then via further updates through 2025.
