In scientific stories, artificial intelligence often becomes all kinds of smart, competent and sometimes suicide robots. A revealing limitation of the best AI of today is that it is for the time being trapped squarely in the chat window.

Google Deepmind has indicated a plan to change that today – probably minus the murder death – by announcing a new version of its AI model Gemini that merges language, vision and physical actions to stimulate a series of more competent, adaptable and potentially useful robots.

In a series of demonstration videos, the company showed several robots with the new model, called twin robotics, which manipulated items in response to spoken assignments: robotic arms, handed over vegetables, gently put a few glasses in a case and completed other tasks. The robots rely on the new model to connect items visible to possible actions to do what is being told to them. The model is trained in a way through which behavior can be generalized across different hardware.

Google Deepmind also announced a version of its model called Gemini Robotics-Mer (for embodied reasoning), which only has visual and spatial understanding. The idea is that other robotic researchers should use this model to train their own models to control robots’ actions.

In a video demonstration, Google Deepmind researchers used the model to control a humanoid robot named Apollo, from the boot AppTRONIK. The robot talks to a person and moves letters around a tabletop as they instruct.

“We were able to bring the world concept-the general concept of twin 2.0 to robotics,” said Kanishka Rao, a robotic researcher at Google Deepmind who led the work, during a briefing before the announcement of today.

Google Deepmind says the new model can successfully control different robots in hundreds of specific scenarios that have not previously been included in their education. “Once the robotic model has a general concept, it becomes much more common and useful,” Rao said.

The breakthroughs that led to powerful chatbots, including Openai’s chatgpt and Google’s Gemini, have raised the hope of a similar revolution in robotics over the past few years, but major obstacles remain.