Two weeks ago, OpenAI collaborated with Figure to create next-generation AI models for humanoid robots. Now Figure has dropped Figure 01’s new status update video, which shows the fascinating capabilities of the robot after its collaboration with OpenAI’s AI models.

Figure 01’s New Status Update Video is Insane

Figure 01’s New Status Update video showed how the robot was able to tell what it saw. The robot said that there’s an apple in the center of the table, a drying rack with cups and plates, and the person’s hand on the table. When the person asked Figure 01, “Can I have something to eat?” Figure 01 gave him the apple present on the table.

The person then put trash on the table and a trash basket on the side and asked the robot, “Can you explain why you did what you just did while you pick up this trash?” Figure 01 successfully replied to the person, “I gave you the apple because it is the only edible item I can provide you with from the table,” and simultaneously, it was putting the trash in the trash basket.

The person then asked the robot, “Based on the scene right now, where do you think the dishes in front of you go next?” Figure 01 replied, “The dishes on the table, like that plate and cup, are likely to go into the drying rack next.” The person replied, “Great, can you put them there?” and Figure 01 completed the job successfully.

In the end, it was asked, “How do you think you did?” Figure 01 replied, “I think I did pretty well; the Apple found its new owner, the trash is gone, and the tableware is right where it belongs.”

Technical Details Behind Figure 01’s New Status Update

Figure and OpenAI’s collaboration helped Figure 01 do the things shown in the video. OpenAI provides visual reasoning and language understanding, whereas Figure’s neural networks are able to carry out fast, low-level, dexterous robot actions.

Figure’s AI lead, Corey Lynch, explained the technical workings behind Figure 01 in a thread on X. He said the Figure 01 shown in the status update video is not teleoperated. He said that we feed images captured from Figure 01’s camera to a large multimodal model trained by OpenAI that understands both images and text. This model is responsible for converting speech to text after capturing it from the robot’s onboard microphone. 

The model then processes the entire history of the conversation, including past images, to come up with language responses that are spoken back to humans via text-to-speech. 

“The same model is responsible for deciding which learned, closed-loop behavior to run on the robot to fulfill a given command, loading particular neural network weights onto the GPU, and executing a policy,” said Lynch.

Impressive Things from Figure 01’s New Status Update Video

The most impressive things that came out of Figure 01’s new status update video were:

• Figure 01’s voice sounded very much like a human, with pauses, emotion, and affection. The founder of Figure, Bret Adcock, said that the voice is a neural net.

• Figure 01 can now reflect on its memory with the help of OpenAI’s multimodal model, as in the video when it was asked, “Can you put them there?” The model can do this only if it knows what “them” and “there” mean. For this, the model needs to reflect on memory and analyze the conversation history.

• Figure 01 can now describe the visual appearance, and the visual and language reasoning is faster than expected.

• There is no teleop, so the robot was not remotely controlled and was able to complete all the tasks autonomously.

• The video was filmed continuously at 1x speed, so you can see that there is a dramatic speed-up of the robot, and Adcock says we are starting to approach human speed.

Conclusion

It has only been 13 days since OpenAI and Figure joined hands, and we are able to see such an amazing result. It looks like it will be very hard for Elon Musk to beat them now. The way Figure 01 is able to do tasks with perfection is fascinating to watch, and when it gets integrated with GPT5, it will be interesting to see the capabilities of Figure 01.

Read More

Cognition AI’s Devin: An AI Software Engineer that Creates Websites and Apps with Simple Prompts

Now Create Consistent Character Images in Midjourney: Here’s How