Meet Happy Oyster: Alibaba launches new AI video model that turns text prompts into playable 3D worlds

Alibaba has launched Happy Oyster, a new artificial intelligence “world model” capable of generating 3D environments and interactive videos that simulate real-world physics. The new AI model announcement comes shortly after OpenAI decided to shut down its Sora video generation service. Meanwhile, Alibaba’s new model also faces competition from Tencent, whose Humayun HY-World 2.0 model is also capable of reconstructing full 3D worlds from video clips.

Notably, Happy Oyster had earlier gone viral after the model, under the pseudonym HappyHorse-1.0, had topped the benchmark platform Artificial Analysis’ text-to-video and image-to-video rankings. The company later revealed that Happy Horse was indeed made by Alibaba, and the model later also made an appearance on other benchmarking platforms.

Explaining the appeal of the new model, Alibaba said, “Unlike traditional AI video tools that follow a one-shot workflow of ‘write a prompt, wait for rendering, receive a finished clip’, Happy Oyster keeps listening and responding throughout the generation process. The scene reacts in real time and continues to evolve as the user keeps giving instructions.”

Why is Happy Oyster special?

Alibaba explained in its official blog that Happy Oyster operates through two core modes: Directing and Wandering.

Rather than just generating standard footage, the Directing mode creates a “running physical world where lighting, gravity, character motion, and scene causality remain continuous and coherent over time,” the company explained in its blog post. Alibaba says users can also step in at any moment using text, voice, or image prompts to switch camera angles, direct characters, or alter the storyline in real time.

The Chinese tech giant said in its blog post that the model’s wandering mode allows users to build and explore an infinitely extendable world from a single text or image prompt. The company says users can roam the generated environment in a first-person perspective, controlling their movement with standard WASD keyboard and camera controls while the world remains stable and continuous.

“Users can roam in first-person, freely change direction and camera movement, and move beyond the original frame while the world keeps generating and remains coherent,” the company said in its blog post.

Alibaba says both modes support multimodal inputs and can generate full audio and video outputs. Directing mode can generate up to three minutes of continuous video at 480p or 720p resolution. Meanwhile, Wandering mode is currently limited to one-minute generations at 480p resolution.

What can Happy Oyster be used for?

Alibaba says its new model can be used for real-time film production, rapid storyboarding, and creating interactive short series where “viewer choices make every watch unique”. For the gaming sector, the company says Happy Oyster can be used to turn prompts into playable worlds with their own consistent logic and lore for concept development.

Alibaba says that the model is available on ‘early access’ by registering for a waitlist on happyoyster.cn