What Is Virtual Production?
Virtual production combines physical and virtual filmmaking techniques to create cutting-edge media. How it works: teams use real-time 3D engines (game engines) to create photorealistic sets, then display them on large LED walls behind physical sets using the real-time rendering capabilities of the game engines. The cameras are synced with the game engines for enhanced realism and depth of perspective.
With virtual production, visual effects are no longer applied to a scene only in post-production. Now, they are worked and iterated on throughout pre-production (known as previs). The old mantra of “fix it in post” is now becoming “fix it in pre.”
For studios of all sizes, there are a lot of advantages for adopting virtual production techniques.
Virtual Production Advantages
The media and entertainment industry’s recent push to virtual production is partially due to the pandemic. The sudden shift to remote work and social distancing meant coming up with a different way to get work done fast while fewer team members were on set. It required film crews to adopt new tools and adjust their creative workflows and pipelines.
But virtual production is about more than just connecting virtually. It has uncovered several advantages that allow studios to build bigger, innovate more, and deliver high-quality results earlier in the production process.
Time and Cost Savings
For many studios, virtual production saves time and money. Virtual scouting — creating virtual versions of proposed sets — can eliminate travel time and additional expenses. Making creative decisions and revisions earlier in the process ensures that the right decisions can be made while actors are on set. This leads to cleaner takes and less rework. It also saves a ton of time (and money) in post-production.
Virtual production allows creatives to develop their vision earlier. It opens room for exploration by visualizing shots and environments before anyone steps foot on set. Iterating on their vision helps studios bring people together to create something out of this world.
Historically, filmmaking has been siloed. It wasn’t until the final release that the rest of the crew could see the CGI graphics and re-edited storylines. But with virtual production, everyone can have a more complete understanding of the film, from previs to post.
Virtual cameras and green screen live compositing act as a window into virtual worlds so contributors can see exactly what they are capturing. LED walls show filmmakers and actors what the set they are on looks like, both through their eyes and in-camera. (Two major added benefits to having graphics displayed on LED screens behind the actors: the lighting it casts on the foreground set and actors is more realistic, and actors can more authentically react to the set.)
Quick On-Set Transitions
Using a virtual environment, teams can transition from one scene to the next with minimal interruption. All of it is happening at one location! Digital assets can be changed up quickly, allowing filmmakers to shoot several scenes with the same actors.
Also, in a virtual location, the filmmaker has full control over the weather, time of day, lighting, object position, and more. These conditions can be recreated at any time and are immediately available.
Library of Assets
Virtual production teams can utilize pre-built assets from existing libraries. They can quickly be customized to meet the needs of any production, without having to build from scratch. These virtual assets are stored electronically, reducing the cost of physical storage, transportation, and labor.
Virtual content can now be created by decentralized teams around the globe without delays. Now filmmakers can build up their teams with creatives and technical specialists no matter where they are located. The possibilities widen without increasing the cost.
How to Build a Virtual Production Studio
Transitioning to a virtual production workflow requires a different way of thinking about the production process. It is less linear and involves new tools — like game engines.
The flexibility and interactivity of virtual worlds within the game engine, combined with the ability to visualize content earlier on in the production process, uncovers new possibilities for studios to realize and refine their vision.
Films like Lion King, Ripple Effect, and Mandalorian were made using game development technology. Just like creating a video game, using a game engine for virtual production requires a high level of orchestration. It is not a linear process, and for that reason, all teams need much more information earlier in the production process. They also need an array of tools to help them iterate on their ideas throughout the process.
What VFX Studios Need for a Virtual Production Pipeline
How can you set up your virtual production pipeline? What do you need to get started? Review some the basic virtual production tools and learn how to get started.
Real-Time 3D Engines/Game Engines
Real-time 3D engines/game engines are essential for virtual production. With tools like Unreal Engine and Unity, teams in media & entertainment create photorealistic virtual worlds and characters to be displayed on LED walls on set in real-time. Then, the practical cameras on set are synced with virtual cameras in the game engine, so that the virtual scene moves as the cameras do, offering enhanced realism and depth of perspective.
With the highly anticipated release of Unreal Engine 5, studios will be able to expand their worlds and adjust sounds, animation, and lighting in just seconds.
Learn how to leverage your game engine >>
Note on Game Engine Integrations
Your tools need to be able to integrate with each other to support other designers and artists. Photoshop, Maya, and 3DS Max are also used to build assets. Pulling all of these extremely large files together using Unreal and Unity makes the final film. Whatever tools you end up using, game engine integrations should be a primary focus. Learn more about:
- Unreal Virtual Production
- Unity Virtual Production
Complete Guide to Game Engines
Learn more about game engines and get started fast.
GAME ENGINE OVERVIEW
Cloud Data Management
When people are working on lots of large files, they need instant access. Moving to the cloud lets more people collaborate wherever they are located.
Moving to a hybrid or cloud model can help teams access the virtual production assets they need quickly. Cloud data centers are available everywhere and can quickly be spun up based on where teams are located. They get faster throughput and lower latency. This avoids the WAN wait that can quickly delay your film. Then you can give people permissions to these files and track how they evolve.
Studios who adopt game engines for virtual production find themselves with massive digital files to manage, increasingly complex projects, and large, often dispersed teams. How can they manage all of that? With a robust version control system — a tool that tracks and manages changes to files over time.
Good version control enables teams to contribute to the same project without overwriting, breaking, or duplicating each other’s work. It also allows teams to share massive files to remote team members instantly. Game developers have been using version control for decades, and so filmmakers are beginning to use it as they adopt the same technology.
Learn how version control helped Final Pixel streamline asset creation and accelerate their virtual production pipeline. >>
Digital Asset Management
Virtual production involves creating many digital assets. These binary art files are often massive, making them difficult to manage. That is where version control like Helix Core comes in. But many creative teams also opt to use digital asset management (DAM), which make it easier to manage all of their digital creative files.
A lot of these tools only store the final version of a file, though, which isn’t helpful when the team needs to find or revert to an older version. Helix DAM, digital asset management by Perforce, solves for this issue and more. Built on Helix Core, it keeps a complete history of every art asset in a visual library. AI-generated tags paired with a smart search tool make it easy for creative teams to find exactly what they need, when they need it. Furthermore, it ensures the final, approved asset is what ends up on the LED volume — meaning it helps teams avoid costly reshoots and reduces time spent editing in post.
Getting Started with Virtual Production
From pre to post-production, software is necessary to help propel the innovations of virtual production pipelines. The learning curve for this method of filmmaking can be steep. That’s why Perforce has teamed with virtual production studio ICVR to bring you free, on-demand, and in-depth virtual production courses.
Perforce U College of Virtual Production courses help anyone — from creatives to developers to managers — master virtual production tools, concepts, and workflows. Become a certified virtual production pro at your own pace with courses on:
- Virtual Production Basics
- Version Control for Virtual Production
- Unreal Virtual Production & On-Set Workflows
- And More