World Labs, the startup founded by AI pioneer Fei-Fei Li, has revealed his first project: an AI system that may generate video game-like 3D scenes from a single image.
Many from AI systems can convert a photograph into 3D models and environments. But World Labs' scenes are unique in that they’re interactive and modifiable.
“(Our technology) permits you to immerse yourself in any image and explore it in 3D,” World Labs wrote in a blog post. “Everything beyond the input image is generated.”
The AI-generated scenes, which anyone with a keyboard and mouse can explore in a demo on World Labs' website, look impressive, if just a little cartoonish. They are rendered live within the browser and have a controllable camera with adjustable simulated depth of field (DoF). The stronger the DoF effect, the blurrier background objects appear.
World Labs' system is a component of an emerging AI category called “world models.” Many from this Models can simulate games and 3D environments – but with artifacts and consistency problems. For example, the Minecraft-simulating world model Oasis from the startup Decart has a low resolution and quickly “forgets” the arrangement of the degrees.
In contrast, World Labs' approach ensures that scenes remain the identical once created and obey the fundamental laws of physics, meaning they’ve a way of solidity and depth.
World Labs' system can even apply interactive effects and animations to scenes, equivalent to changing the colour of objects and dynamically lighting backgrounds.
“Most generative AI tools create 2D content equivalent to images or videos,” wrote World Labs. “Instead, generating in 3D improves control and consistency. This will change the best way we create movies, games, simulators and other digital manifestations of our physical world.”
Now there may be definitely room for improvement. World Labs' scenes aren’t fully explorable – your movements are limited to a small area. (Try to maneuver beyond that and also you'll hit a limit.) And there are occasional rendering errors – for instance, when objects mix into each other in unnatural ways.
But World Labs says it's just an “early preview.”
“We are working hard to enhance the dimensions and fidelity of our generated worlds and to experiment with recent ways for users to interact with them,” the startup wrote within the blog.
World Labs, which launched earlier this yr, has raised $230 million in enterprise capital from investors including Andreessen Horowitz (a16z), Ashton Kutcher, Intel Capital, AMD Ventures and Eric Schmidt. The company is valued at over $1 billion and hopes to finish its first product in 2025.
Beyond interactive scenes, World Labs plans to develop tools that could possibly be useful for professionals equivalent to artists, designers, developers, filmmakers and engineers. It targets customers starting from video game developers to film studios.
“We have already got the power to create virtual, interactive worlds, but that costs a whole bunch and a whole bunch of hundreds of thousands of dollars and a variety of development time,” Justin Johnson, co-founder of World Labs, said in a recent press release Consequence of the a16z podcast. “With (world models) you get not only a picture or a clip, but a completely simulated, living and interactive 3D world.”
