He is trained on hits
As we already understood from the example of GPT-4, a neural network that is not trained to play games does not cope with this task very well. Therefore, Google decided to teach artificial intelligence to play games.
DeepMind spoke about its work on the Scalable Instructable Multiworld Agent (SIMA) — universal gaming AI agent.
In fact, this is a neural network that learns to play games. And with very specific examples. The developers of such hits as No Man's Sky, Teardown, Valheim, Goat Simulator 3 and others cooperate with Google. SIMA's creators also used four exploration environments, including the new Unity-based Construction Lab environment, where agents have to build sculptures from building blocks that test their manipulation of objects and intuitive understanding of the physical world.
Google focuses on the fact that SIMA is not taught to win in certain games. This AI is rather created as a potential partner for real gamers. For example, in some kind of conventional cooperative survival game, an artificial companion could be tasked with collecting resources or building some kind of fortifications.
Importantly, SIMA is built in such a way that it does not require any special API or other access to the game. AI only needs images from the game and commands in natural language. That is, in theory, it can be connected to any game.