News provided by:’ Kunlun Tech , Aug 19, 2024, 08:13 ET
SINGAPORE, Aug. 19, 2024 /PRNewswire/ — On August 19, Kunlun Tech launched SkyReels, the world’s first AI reels platform integrating AI video generation model with 3D engine. SkyReels integrates script generation, character customization, storyboarding, plot development, dialogue/BGM, and film synthesis into a single seamless experience, allowing creators to easily generate high-quality AI reels and videos.
Whether it’s epic fantasy, werewolf/vampire romance, or family comedy, SkyReels can transform a simple idea into a complete reel.
SkyReels combines Kunlun Tech’s script-writing Large Language Model (LLM) SkyScript, storyboard LLM StoryboardGen, 3D generation LLM Sky3DGen, and the innovative WorldEngine platform that integrates AI 3D engines with AI video generation models for unprecedented creativity.
SkyReels excels in script quality, storyboarding, and character performance. The platform generates full scripts, storyboards, character dialogues, and BGM with a single AI-powered click. It supports customization of character visuals, voices, and storyboards, and automatically converts content into 1080P 60fps HD video. It can generate videos up to 180 seconds long, a significant improvement over other platforms. This all-in-one integration dramatically enhances video creation efficiency, reduces costs, and accelerates the era of “one person, one film.”
WorldEngine combines precise control capabilities (such as lighting simulation, physics simulation, 3D space, real-time interaction) with the imaginative generation capabilities of AI video models, offering a new online hybrid video creation mode. This allows video creation to move from vague generation to precise control.
Highlights of SkyReels:
- Fully Automated Script Generation: Users input a concept or idea, and AI will generates a complete, well-structured, and engaging script with a single click. It also supports automatic organization and refinement of existing scripts to boost efficiency.
- High-Quality Plot Design: The system analyzes a vast amount of script data to generate high-quality storylines, dramatic conflicts, engaging points, and emotional expressions, ensuring richness and appeal.
- Intelligent Storyboarding: Automatically generates storyboard images and corresponding text scripts based on the script. Users can adjust storyboards, camera angles, and movements to achieve precise creative implementation.
- Emotional Dialogue/BGM: Using emotional voice synthesis technology, characters receive natural pauses, emotional expressions, and intonation. AI-generated BGM is automatically created based on video content for perfect integration of visuals and music.
- Personalized Customization: Supports intelligent generation and customization of character visuals, voices, and storyboards. Offers various visual style options, including anime, cartoon, and realistic 3D, to meet different creator preferences and content needs.
- Consistency and Precision: Ensures consistency of characters and scenes across different storyboards while automatically deriving character performances based on the story. Enhances the coherence of expressions, actions, and dialogue to improve storytelling.
- Efficient Video Generation: Automatically converts storyboards into continuous HD videos, providing 1080P 60fps output for a smooth viewing experience.
- Convenient Post-Production: Integrates all creative outputs with one click, supports quick export and sharing in multiple formats, and facilitates easy post-production editing.
SkyReels’ AI script generation is powered by Kunlun Tech’s SkyScript LLM. To ensure the AI-generated scripts feature high-density plot points, Kunlun Tech developed the SkyScript-100M dataset, which provides high-quality annotations for script pacing, plot points, and emotional shifts.
The algorithm team designed a progressive script generation framework to produce highly structured and engaging scripts, enabling both novice and experienced writers to easily create captivating reel scripts.
Another crucial aspect of reel creation—storyboarding—is handled by Kunlun Tech’s StoryboardGen LLM. StoryboardGen, based on the DiT-MoE architecture, generates high-density storyboard information, greatly enhancing visual storytelling.
After inputting the script, StoryboardGen breaks it down into two parts: global and main. The global part includes the overall description of the storyboard, such as Scene, Caner and Layout, while the main part involves customization of character and actions. This is then compiled into a complete storyboard by the StoryboardGen intelligent agent. The specialized multi-agent architecture of StoryboardGen ensures controllable and consistent characters and scenes, producing high-quality, industrial-grade storyboards.
WorldEngine is the industry’s first innovative platform to deeply integrate AI 3D engines with AI video generation models. This groundbreaking technology uses layer fusion to create an unprecedented 3D-video hybrid creation mode. Users can choose to use either the 3D AI engine or video large models for any element. This innovative approach overcomes common issues in current AI video models, such as “anti-gravity” effects, achieving a perfect blend of physical simulation precision and imaginative creativity of video models.
Regarding 3D AI engines, Kunlun Tech’s Sky3DGen 3D generation LLM diverse 3D elements and scenes, from natural landscapes to futuristic cities, and from static objects to dynamic characters. It also provides real-time 3D scene interaction, allowing users to see the effects of adjustments and edits instantly, thus speeding up the creation process. With WorldEngine, users can easily construct 3D virtual studios, achieving a dream-like virtual shooting experience.
Character performance is another core aspect of reels presentation. Thus, Kunlun Tech’s SkyReels team developed the ActorShow LLM for AI character performance, enhancing the controllability of lip-sync expressions and body movements, making the characters appear more natural.
To address issues such as lighting and style discrepancies in video post-production, Kunlun Tech’s SkyReels team has trained a specialized fusion generation model to ensure consistent output after layer fusion.
The launch of Kunlun Tech’s AI reel platform, SkyReels, represents a technological leap in global AI reel creation. It will revolutionize video content creation processes, freeing creators from complex workflows and focusing on creative realization and inspiration, bringing the dream of “one person, one film ” closer to reality.
SkyReels not only provides powerful creation tools for professional content creators but also significantly lowers the barriers to AI reel creation, allowing non-professional users to easily get started. This revolutionary tool is expected to drive explosive growth in both AI reels user-generated content (UGC) and professional user-generated content (PUGC), injecting unprecedented vitality into the reel content ecosystem and accelerating the rapid growth of the reel content creation and consumption market. An era of AI-driven reel creation and consumption is on the horizon.
SOURCE: Kunlun Tech