Google researchers deal a major blow to the theory AI is about to outsmart humans

• The race to build AI as smart as humans, or AGI, looks like it suffered a major blow.
• Google researchers found the transformer technology behind AI isn't very good at generalizing.
• "We shouldn't get too crazy about imminent AGI at this point," one AI expert told Insider.

Google researchers may have just given a major reality check to the ambitions of CEOs in chase of AI's holy grail.

In a new pre-print paper submitted to the open-access repository ArXiv on November 1, a trio from the search giant found that transformers - the technology driving the large language models (LLMs) powering ChatGPT and other AI tools - are not very good at generalizing.

"When presented with tasks or functions which are out-of-domain of their pre-training data, we demonstrate various failure modes of transformers and degradation of their generalization for even simple extrapolation tasks," authors Steve Yadlowsky, Lyric Doshi, and Nilesh Tripuraneni wrote.

What transformers are good at is performing tasks that relate to the data they've been trained on, according to the paper. They're not so good at dealing with tasks that go even remotely beyond that.

That's a bit of a problem for those hoping to achieve artificial general intelligence (AGI), a term techies use to describe hypothetical AI that can do anything humans do. As it stands, AI is pretty good at specific tasks but less great at transferring skills across domains like humans do.

It means "we shouldn't get too crazy about imminent AGI at this point," Pedro Domingos, professor emeritus of computer science and engineering at the University of Washington, told Insider.

AGI has been touted as the ultimate goal of the field of AI because it represents the moment, in theory, when humanity creates something that is as smart as, or smarter than, itself. Depending on your point of view, it's an alarmingly Promethean or an era-defining scenario. Either way, a lot of investors and techies are putting serious time and investment into getting there.

Standing on stage with Microsoft CEO Satya Nadella on Monday, for instance, OpenAI boss Sam Altman reiterated his desire to "build AGI together."

Achieving that means getting AI to do a lot of the generalizing tasks that they human brain can do — whether it's adapting to unfamiliar scenarios, creating analogies, processing new information, or thinking abstractly.

But if the technology struggles with even "simple extrapolation tasks," as the researchers note, clearly we are not close yet.

"This paper isn't even about LLMs but seems to be the final straw that popped the bubble of collective belief and gotten many to accept the limits of LLMs," Princeton computer science professor Arvind Narayanan wrote on X. "About time."

Jin Fan, senior AI scientist at Nvidia, questioned why the paper's findings were a surprise to people as "transformers are not elixirs."

The research highlights how "a lot of people have gotten very confused" about the potential of a technology being touted as a path towards AGI, said Domingos.

"This paper that just came out, it's interesting who it's surprising to and who it's not surprising to," he added.

Though Domingos acknowledges transformers are an advanced technology, he believes a lot of people think they're a lot more powerful than they actually are.

"The problem is that neural networks are extremely opaque and also these LLMs have been trained on unimaginably large amounts of data which got a lot of people very confused about what they can and can't do," he said. "They start thinking they can do miracles."

More advanced forms of AI may do a better job of generalizing. The Google researchers used a GPT-2 scale model rather than something more current like a GPT-4 scale model.

Sharon Zhou, CEO of Lamini AI, told Insider she doesn't find it troubling that transformers may struggle to generalize.

"It's why I started a company that trains models, not just queries them, so it can learn new things," she said. They can still be very useful, and still be steered and aligned."