Imagine a sci-fi movie in which an AI can suddenly do things that nobody expected, that weren’t even programmed in by its own developers.
Plot twist: This scenario is not science fiction, but everyday life when dealing with large AI models.
What happens when language models like GPT-3 and PaLM, with their hundreds of billions of parameters, surprise us with completely unexpected capabilities? Consider these examples:
- A chatbot becomes a Linux terminal.
- Some AI models suddenly solve problems like MacGyver.
And what does this have to do with the „Double Descent“ phenomenon? Do AI systems really always get worse before they suddenly become brilliant?
Just as a single atom of gold does not shine golden, completely new, emergent properties only develop in AI once a critical mass has been reached. My latest entry in the AI logbook takes you on a journey through the world of emergence in AI systems and the breakthroughs that may still await us in 2025.