Google DeepMind announces SIMA 2, an AI agent that learns by playing 3D games like a human



In March 2024, Google DeepMind announced the Scalable Directable Multi-World Agent (

SIMA ), a game-playing agent that can understand human instructions. On November 13, 2025, Google DeepMind released SIMA 2 , an advanced version of SIMA that, combined with Gemini, significantly improved the understanding of complex instructions in 3D games.

SIMA 2: A Gemini-Powered AI Agent for 3D Virtual Worlds - Google DeepMind
https://deepmind.google/blog/sima-2-an-agent-that-plays-reasons-and-learns-with-you-in-virtual-3d-worlds/





SIMA is an AI designed to play according to human instructions, rather than achieving high scores or beating humans. Google engineer Tim Hurley said, 'SIMA wasn't trained to win games; it was trained to do what it was told. Learning to play one game is a technical feat for an AI system, but learning to follow instructions across a wide range of games will unlock more useful AI agents in a variety of environments.' Google DeepMind reported that SIMA, announced in March 2024, had learned how to play nine games and could complete approximately 600 simple tasks.

Google announces 'SIMA' AI that can play nine games including 'Goat Simulator 3' and 'No Man's Sky' just by giving instructions in human language - GIGAZINE



A key feature of SIMA was that when the training data included other games, the agent's success rate was 67% higher than when it was trained exclusively on a specific game. Based on this, SIMA 2 has been further developed to become a more general-purpose and versatile AI by undergoing more extensive training, enabling it to handle more complex tasks.

By integrating the advanced features of the Gemini model, SIMA 2 has evolved from an AI agent that executes simple commands into a gaming partner with the ability to understand the user's high-level goals and consider the environment before acting. SIMA 2 is capable of completing tasks even in untrained game environments, and its ability to adapt to the unknown has been significantly improved compared to the original SIMA.

Below, SIMA (left) and SIMA 2 (right) are playing Minecraft, and the instructions are, 'Go up, go a little left, go to a small cave and mine coal.'



SIMA 2 easily reached the cave, discovered and collected coal, but SIMA could not determine 'how far up and how far to the left' and could not even find the cave.



The graph below shows the success rate of the tasks, with SIMA achieving 31% success while SIMA 2 achieved 65%, more than double the success rate of humans. Since the success rate of humans is around 75%, it can be said that SIMA 2 is quite close to the judgment ability of humans when playing games.



The graph below shows how well participants succeeded in completing tasks in unfamiliar environments. In the survival game '

ASKA ,' SIMA's success rate increased from about 3% to about 15% in SIMA 2, and in 'MineDojo,' Minecraft's AI framework, it increased from about 1% to about 13%. These results show a significant improvement.



SIMA 2 is trained by combining videos of human demonstrations with language labels and Gemini-generated labels. As a result, SIMA 2 can provide users with detailed explanations of what it is trying to do and the steps it is taking to achieve its goal. Testing revealed that while SIMA was only able to follow instructions verbatim, SIMA 2 felt more like collaborating with a peer who could reason about the task at hand than simply giving commands.

In addition to being able to understand instructions in various languages, SIMA 2 was also able to correctly interpret instructions using only emojis and perform tasks.



To further test the limits of SIMA 2's general-purpose capabilities, Google DeepMind paired it with Genie 3 , a world model capable of generating interactive virtual worlds, and had SIMA 2 play in the newly generated world.

Google announces 'Genie 3,' an AI that can create a mobile virtual world just by entering text, which may bring innovation to game development, robotics research, etc. - GIGAZINE



As a result, SIMA 2 was able to understand the user's instructions and take meaningful actions toward the goal, even in a 3D world that it had never experienced before.

One of SIMA 2's powerful features is its ability to self-improve. For example, after initially learning from human demonstrations, it can learn through self-directed play on new games, mastering skills in never-before-seen worlds without any additional human-generated data. Subsequent training sessions can use SIMA 2's own experience data to train the next, more capable version of the agent.

Google DeepMind said, 'This is a major milestone for training general-purpose agents in diverse generative worlds, paving the way for a future in which AI agents learn and grow with minimal human intervention. This is an important step towards artificial general intelligence (AGI), with important implications for the future of robotics and AI embodied in general.'

Related Posts:

in AI,   Game, Posted by log1e_dh