OpenAI introduces a new text-to-video AI generator known as Sora. OpenAI’s Sora can create up to 60-second videos with highly detailed scenes, complex camera models, and multiple characters. The model also knows how things asked in a prompt actually exist in real life. OpenAI’s Sora can also generate videos from still images by animating the image’s contents. The model is also capable of extending an existing video or filling in missing frames.
Sora is a diffusion model, and it was trained using publicly available videos and some copyrighted videos licensed for the purpose; however, OpenAI has not mentioned the exact sources. Sora generates entire videos all at once, which makes sure that the subject stays the same even when it goes out of view temporarily. Sora uses the recaptioning technique from DallE3, which interprets a user’s prompt and creates highly descriptive captions that are used by the AI model to generate videos.
The technology has not yet been released, but it is available to only a limited number of visual artists, designers, and filmmakers. The company will take feedback from them and work to advance the AI model.
Steps taken by OpenAI to ensure Security with Sora
With AI video-generating models, security has always been an issue, so OpenAI is taking important safety steps to ensure Sora is not used for ill purposes.
To test their model, they are working with red teamers who are experts in areas like misinformation, hateful content, and bias. They will be adversely testing the model to identify risks associated with it. OpenAI is also building tools that can detect whether a video was generated by Sora, and it also plans to add C2PA metadata in the future.
Just like DallE-3, Sora will also use the existing safety methods, like the text classifier, which checks and rejects input prompts that violate OpenAI’s usage policies. This includes the prompts that request extreme violence, sexual content, hateful imagery, and celebrity likeness.
OpenAI has also developed robust image classifiers that will check the frames of every generated video to make sure it adheres to their existing usage policies.
Current Weaknesses of OpenAI’s Sora
As the model is not yet released and is currently in the testing stage, there are some issues encountered while using it. OpenAI mentioned this in their blog post: “It may struggle with accurately simulating the physics of a complex scene, and may not understand specific instances of cause and effect. For example, a person might take a bite out of a cookie, but afterward, the cookie may not have a bite mark.” OpenAI also says that it sometimes confuses the spatial details of a prompt.
Videos generated by OpenAI’s Sora
OpenAI took to X and posted a series of AI videos generated solely by Sora. You can also see the sample videos generated by OpenAI’s Sora on their website. OpenAI has posted a series of sample videos along with their prompts. The video samples have shocked the world. The generated videos look highly realistic, and it can’t be said that they were created by AI. It is difficult to tell if these are not AI-generated.
Prompt: “A stylish woman walks down a Tokyo street filled with warm glowing neon and animated city signage. she wears a black leather jacket, a long red dress, and black boots, and carries a black purse. she wears sunglasses and red lipstick. she walks confidently and casually.… pic.twitter.com/cjIdgYFaWq
— OpenAI (@OpenAI) February 15, 2024
Conclusion
This is going to change how filmmakers and content creators create their videos. The AI-generated videos look highly realistic, and it is difficult to tell whether they are AI-generated. The possibilities of Sora are endless, but it can also be used to generate misleading videos. It will be a big headache for OpenAI to control the ill practices of their tool. So it is important for OpenAI to first fully establish security and privacy norms before deploying this model for everyone. It is only February, but it seems that OpenAI’s Sora is already one of the biggest things of 2024.
Read More
0 Comments
Trackbacks/Pingbacks