‘The world abounds with quantities’ — DeepMind is more ambitious than ever

Apr 18

Written By Center for AGI Investigations

We’re not sure what this is, but it’s exciting

You got to hand it to DeepMind. They’re the Apple of laboratories. While everyone else is clamoring to occupy space in the public head, it’s toiling mysteriously in the background.

Just as the entire world was getting ready to hold its breath in anticipation of ChatGPT 5.0 or its “pre-AGI” equivalent from Anthropic or Google, DeepMind casually intimates that everyone’s been going about this whole “AI” thing all wrong.

"Where do rewards come from, if not from human data?”

DeepMind researchers David Silver and Richard Sutton recently published a paper which is, evidently, linked to an upcoming book on the subject of AI agency.

The gist of the work is that, in its current large language model form, AI has no agency. If we put AI models in the real world, and give them the ability to continuously interpret that world, we’ll be able to get around the whole single player gaming experience problem with chatbots.

Today, talking to an AI is like playing an open-world video game by yourself. Your experiences with chatbots don’t matter to anyone else because the world they exist in isn’t a shared one.

To get around this problem, DeepMind wants to take the idea of “reward-based behavior” and expand it beyond the digital realm and into the physical world. Per the paper:

“Once agents become connected to the world through rich action and observation spaces, there will be no shortage of grounded signals to provide a basis for reward. In fact, the world abounds with quantities such as cost, error rates, hunger, productivity, health metrics, climate metrics, profit, sales, exam results, success, visits, yields, stocks, likes, income, pleasure/pain, economic indicators, accuracy, power, distance, speed, efficiency, or energy consumption. In addition, there are innumerable additional signals arising from the occurrence of specific events, or from features derived from raw sequences of observations and actions."

We’ll let you draw your own conclusions. The paper is definitely worth a read.