Google’s SIMA, short for Scalable, Instructable, Multiworld Agent, is an AI agent from Google Deepmind that understands and follows natural language instructions in 3D environments and video games. It has the ability to learn from various video games and 3D environments. It can play a game with you as a companion, and you can instruct the SIMA in natural language based on what you want it to do in the video game.
For example, instructing it to pick up leaves or driving a car in a video game.
SIMA is trained on a large and diverse dataset of gameplays; it doesn’t need access to the game’s source code or APIs. It only takes image and text instructions as input, just like humans do. To know more about the technical details, you can view the technical report here.
Google’s SIMA Tested on Various Games
Google has partnered with a lot of gaming studios to train SIMA so that it can interact with various 3D environments and games. Google’s SIMA has been tested across various popular games like No Man’s Sky, Teardown, Valheim, Goat Simulator 3, and more.
SIMA is evaluated on over 600 gaming skills, including navigation such as moving up, down, right, left, driving a car, object interaction such as chop down a tree, climbing ladders, crafting a helmet, and more.
SIMA currently can’t do advanced things that require high-level strategic planning and multiple subtasks, such as finding resources and building a camp. So it is getting trained on complex instructions.
Google’s SIMA is Generalized to Various Games
Unlike specialized AI agents that are developed for a particular game, Sima agents have the ability to generalize to various games. So when the agent faces a new game, it performs tasks based on what it has learned by playing other games.
For example, if SIMA agent plays a FPS game, it can easily play other FPS games, whereas specialized agents can only play that particular game.
“SIMA is trained to run it and do what it’s told; it’s not trained to win a game,” said Tim Harley, a co-lead for SIMA and researcher at Google DeepMind.
When Sima was faced with an unseen environment, it performed nearly as well as the specialized agent, which shows its great generalizing ability.
Availability of Google’s SIMA
SIMA is not being used currently professionally as it is currently in the research phase, and it is being tested and trained.
Conclusion
Google’s SIMA opens up a new space for AI in video games where they can be used as companions rather than just powerful opponents. This also takes AI one step closer to humans, and in the future, it might collaborate with humans by interacting with real-world environments rather than just video games.
Read More
Cognition AI’s Devin: An AI Software Engineer that Creates Websites and Apps with Simple Prompts
0 Comments