Google DeepMind’s New AI Agent Learns, Adapts and Plays Games Like a Human

Google DeepMind launched SIMA 2 on Thursday—a brand new AI agent that the corporate claims behaves like a “companion” inside digital worlds. With the launch of SIMA 2, DeepMind goals to advance past easy on-screen actions and transfer towards AI that may plan, clarify itself, and study by way of expertise.

“It is a vital step within the route of Synthetic Normal Intelligence (AGI), with necessary implications for the way forward for robotics and AI-embodiment usually,” the corporate mentioned on its web site.

The primary model of SIMA (Scalable Instructable Multiworld Agent), launched in March 2024, discovered tons of of fundamental expertise by watching the display and utilizing digital keyboard and mouse controls. The brand new model of SIMA, Google mentioned, takes issues a step additional by letting the AI suppose for itself.

“SIMA 2 is our most succesful AI agent for digital 3D worlds,” Google DeepMind wrote on X. “Powered by Gemini, it goes past following fundamental directions to suppose, perceive, and take actions in interactive environments–which means you may discuss to it by way of textual content, voice, and even photos.”

By utilizing the Gemini AI mannequin, Google mentioned SIMA can interpret high-level objectives, discuss by way of the steps it intends to take, and collaborate inside video games with a degree of reasoning the unique system couldn’t attain.

DeepMind reported stronger generalization throughout digital environments, and that SIMA 2 accomplished longer, extra complicated duties, which included logic prompts, sketches drawn on the display, and emojis.

“Because of this potential, SIMA 2’s efficiency is considerably nearer to that of a human participant on a variety of duties,” Google wrote, noting that SIMA 2 had a 65% process completion charge, in comparison with 31% by SIMA 1.

The system additionally interpreted directions and acted inside solely new 3D worlds generated by Genie 3, one other DeepMind challenge launched final yr that creates interactive environments from a single picture or textual content immediate. SIMA 2 oriented itself, understood objectives, and took significant actions in worlds it had by no means encountered till moments earlier than testing.

“SIMA 2 is now much better at finishing up detailed directions, even in worlds it is by no means seen earlier than,” Google wrote. “It will possibly switch discovered ideas like ‘mining’ in a single recreation and apply it to ‘harvesting’ in one other—connecting the dots between comparable duties.”

After studying from human demonstrations, researchers mentioned the agent switched into self-directed play, utilizing trial and error and Gemini-generated suggestions to create new expertise data, together with a coaching loop the place SIMA 2 generated duties, tried them, after which fed its personal trajectory knowledge again into the following model of the mannequin.

Whereas Google hailed SIMA 2 as a step ahead for synthetic intelligence, the analysis additionally recognized gaps that also have to be addressed, together with fighting very lengthy, multi-step duties, working inside a restricted reminiscence window, and going through visual-interpretation challenges frequent to 3D AI techniques.

Even so, DeepMind mentioned the platform served as a testbed for expertise that might finally migrate into robotics and navigation.

“Our SIMA 2 analysis presents a powerful path in direction of functions in robotics and one other step in direction of AGI in the true world,” it mentioned.