Runway unveiled its latest text-to-video (T2V) generator called Gen 3 Alpha, and demos suggest this might be the perfect AI video generator yet.
OpenAIs Sora got us excited a couple of months ago, but there's no word yet on when (or if) it should be released. Runway already allows free and paid access to its previous generation T2V tool, Gen 2.
Gen 2 makes some decent videos, but is a bit hit or miss and infrequently leads to weird anatomy or clumsy movements when generating people.
Gen 3 Alpha delivers hyper-realistic videos with fluid motion and coherent human models.
Runway says: “Gen-3 Alpha excels at creating expressive human characters with a wide selection of actions, gestures and emotions, opening up latest possibilities in storytelling.”
Introducing Gen-3 Alpha: Runway’s latest base model for video generation.
Gen-3 Alpha can create highly detailed videos with complex scene changes, a wide selection of cinematic selections, and detailed art instructions.https://t.co/YQNE3eqoWf
(1/10) pic.twitter.com/VjEG2ocLZ8
— Runway (@runwayml) 17 June 2024
The improved fidelity also comes with a rise in speed, with the utmost length 10-second clips generated in only 90 seconds. The 10-second clip limit is identical as Sora, double that of Luma, and triple that of Runway Gen 2.
In addition to the improved human representation, the precise physics of the videos are also really impressive.
And considering that this video was 100% AI generated, it's complete insane what we hear about AI videos nowadays. #Runway Generation 3 🔥🔥 pic.twitter.com/FLC5TGfYzr
— Pierrick Chevallier | IA (@CharaspowerAI) 17 June 2024
Runway says Gen 3 Alpha will feature improved control modes that allow a user to pick specific elements to have movement and detailed camera motion controls with “upcoming tools for finer control over structure, style and motion.”
The level of camera control gives an idea of how close we’re to the top of traditional film production.
Prompt: Fast moving handheld camera, flashlight light, black graffiti with the word “Runway” on an old white wall in an old alley at night.
(10/10) pic.twitter.com/xRreX33g0r
— Runway (@runwayml) 17 June 2024
OpenAI has previously suggested that alignment concerns are considered one of the explanations Sora hasn't been released yet. Runway says Gen 3 Alpha comes with numerous latest safeguards and C2PA, which might track the origin of the generated video.
General world models
The idea of turning text into video will appeal to most users, but Runway says Gen 3 Alpha represents a step toward a special goal.
Runway says: “We imagine the following big advance in AI will come from systems that understand the visual world and its dynamics. That's why we're launching a brand new long-term research project around what we call general world models.”
Training an embodied AI to navigate and interact with an environment is far faster and cheaper when simulated. For simulation to be useful, it must accurately represent the physics and motion of real-world environments.
Runway says these general world models “must capture not only the dynamics of the world, but in addition the dynamics of its inhabitants, which incorporates constructing realistic models of human behavior.”
The coherent movements, physics, human features and emotions within the Gen 3 demo videos represent a giant step towards that goal.
OpenAI has almost actually been working on an improved Sora, but with Runways Gen 3 Alpha, the race for the perfect AI video generator has turn into even tougher.
There isn’t any word on when Gen 3 Alpha can be released, but you may You can find more demos here or experiment with Gen 2 Here currently.
