How Virtual Studio Sets Are Changing The Way Movies Are Made
INTRODUCTION
Filmmaking is a creative discipline which is constantly changing and being driven forward by changes in technology.
Whether that’s the change from black and white film to colour, the introduction of anamorphic lenses that led to a widescreen aspect ratio, or the creation of digital cinema cameras and the advances in CGI and post production software which allowed filmmakers to artificially create shots that wouldn’t have been possible before.
Advances in technology have an undeniable influence on filmmaking.
One of those recent advances which I’ll look at in this video is the way in which the film set, the space in which a movie is physically shot, is being altered by improvements in LED wall technology combined with gaming engines. And how we’re quickly heading towards a point where the idea of shooting big budget shows and movies in real world locations is becoming less and less popular.
WHY NOT SHOOT IN REAL LOCATIONS?
If you’ve never been on a film set and don’t know much about how movies are made it may surprise you to find out that on many productions the environments that the characters are filmed in are not actually locations in the real world.
There are two types of filming environments that can be set up, real world places - which is called shooting on location - and fake environments that are artificially created to mimic a space - which is called shooting in a studio.
You may wonder what the point of shooting in a studio is when the real world has no end of beautiful, easily accessible locations. It boils down to a few reasons.
The first considerations are time and money. Even though it’s costly to rent studio space and build a set from scratch, sometimes this is still a cheaper option than shooting on a real location.
For example, some scripts may require multiple scenes shot in a diverse range of interiors. It may be cheaper and easier to build one tent set, one interrogation room set, one office set and one prison cell set next to each other in a studio which the crew can quickly bounce around between, rather than doing multiple hour location moves, or even inter-country moves, between each real world location.
Another more obvious reason to shoot on artificial sets is because it may be impossible, or at least very difficult, to access certain locations in real life. Trying to gain access to shoot in the Oval Office probably isn’t going to go very well.
Thirdly, shooting in a studio gives filmmakers a far higher degree of creative and practical control. When you set lights in a studio they will provide a consistent level of illumination for as long as necessary. When you’re in a real world location the sun will move throughout the day and the quality of the ambient light will constantly change.
When shooting outside in real locations it might rain, there may be clouds or there may be full sun. You’ll constantly have to adapt your plans and scheduling depending on weather forecasts and what kind of look you’re after. This isn’t isn’t an issue when shooting inside a soundstage where you can create your own permanent artificial sun.
Finally, shooting in a studio is sometimes necessary to achieve certain shots with specific gear. For example, doing a telescoping movement through an interior on a Technocrane, or getting a high bird’s eye view perspective, may only be possible in a studio where that gear can practically be brought into the space and where set walls can be moved around, or the set ceiling removed, to accommodate the gigantic rig.
HISTORY OF ‘VIRTUAL’ SET TECHNOLOGY
“Every step that we take in the film business is incremental. Digital didn’t just appear on the scene. It had been precursured with Genesis’ and DVs. It didn’t appear all of a sudden. It feels like it sometimes that the adoption of digital happened overnight. But it actually didn’t.” - Greig Fraser ACS, ASC, Cinematographer
When you compare movies from the 30s and 40s with contemporary films it’s much easier to see which sets are not real in the older films. This background is clearly not real, but what about this one? It may look like a real interior location but this background is actually created by a giant LED screen.
To better understand this cutting edge soundstage of the future it’s best to start at the beginning and go through a brief historical overview of quote unquote ‘virtual set backgrounds’.
One of the earliest ways of creating fake backgrounds in movies was with matte paintings or painted backdrops. This is where an artist was employed to physically paint a landscape or set background onto a sheet of glass. The painting would try to incorporate as much of an illusion of depth as they could using a 2-D surface.
Actors, foreground set design and props were then filmed and placed in front of these painted backdrops to trick the audience into thinking they were at a real location.
To save on the inflexibility, lack of photorealism and lack of camera movement, the next technological step forward used the same idea but replaced it with film projection.
Rear projection, as it was called, used a large screen surface with a film projector mounted behind it that could project photorealistic backgrounds that had been pre-filmed at a real location. This also meant that moving backgrounds could now be projected to give the illusion of motion.
Although this was later improved upon with front projection, it still didn’t always sell these backgrounds as 100% reality.
Moving forward in time to digital effects, the next technological breakthrough came from chroma key compositing. Again, this used a similar principle as before, but instead of painting or projecting a background image that could be captured in camera, this time a consistently coloured blue, or green screen backdrop was used.
Green and blue are the most commonly used background colours for chroma keying as they are uniform, distinct and differ significantly from the hues that are present in human skin and most other human environments.
Using software, this specific green or blue channel of colour can be keyed out and removed from the shot. A secondary shot can then be layered behind this foreground layer in post production, replacing the background with whatever footage they’d like and creating the illusion of depth.
Although this technique has been widely used to create artificial set backgrounds for years, it’s still not perfect. One of the main challenges of shooting with a chroma key is that it does not provide realistic lighting, like a real life ‘background’ in a shot would.
“Cause there’s always the problem.You know, you’re flying above a planet like Earth. If you do a barrel roll how do you suitably light that all around? You’re not going to do a real barrel roll. So trying to solve that problem led us to creative volume.” - Greig Fraser ACS, ASC, Cinematographer
LED VOLUME WALL
Creative volume, or volume lighting, is a way of describing the latest innovation in virtual background technology.
“The stage of the future is a series of lights on the walls. It’s walls that are made of light emitting devices.” - Greig Fraser ACS, ASC, Cinematographer
This is a gigantic LED wall, and sometimes also a ceiling, which can display and playback photo-realistic video or stills using Epic Games’ Unreal gaming engine - kind of like a massive TV. This system can also use camera positional data to change how the background moves. So when the camera moves, the background can move accordingly, creating parallax and an almost perfect visual illusion.
“There’s another shot on that same ice pathway on the ice planet where the camera was booming up. And in camera it’s perfect. There’s one long walkway disappearing. Obviously there was a practical walkway and then the digital wall. And so the digital walkway, as the camera’s booming up, had to change its relationship so that the perspective from the camera was the same.” - Barry Idoine, Cinematographer
This enables most shots to be done completely in camera without much post production tweaking necessary. This wall also solves the lack of interactive lighting problem that’s encountered when using a green or blue screen.
Greig Fraser used this system, which they called The Volume, to shoot large portions of The Mandalorian in studio. Having no green screen meant that there were no green light tinges to the set, or green reflections on the actors metallic suit.
The Volume is a 20 foot high, 270 degree wall with a circumference of 180 feet, complete with a ceiling. This newest iteration of the technology featured LED pixels which were only 2.84mm apart from each other - close enough for it to produce photorealistic backgrounds.
This allows crews to use the gaming engine to map 3D virtual sets as a background using the same technique as early matte paintings or rear projection but with the added bonus of creating realistic parallax movement that mimicked that of the camera movement, and interactive lighting that provided naturalistic shadows, illumination and reflections.
These backgrounds are created by using a series of digital photographs taken on a camera like a Canon 5D which can then be stitched together to create one stretched out background that covers the 270 degree wall.
To change between locations in different cities, or even different planets, the production design crew just needs to swap out the foreground art elements, like the floor and any props near the characters.
The correct background will then be set on the LED wall, any lighting tweaks will be adjusted, the actors called in, and then they’re good to go. This allowed them to be able to change between an average of two different locations in a shooting day.
“Instead of blue, green screen, we can now see the environments and actually see them as live comps. For all intensive purposes. We’ll actually be able to go inside a car on stage and for the actors and the photography to look like you’re actually driving.” - Lawrence Sher, ASC, Cinematographer
One of the big advantages of working like this is that cinematographers can use this LED screen to control the ‘weather’ however they want. If they want to shoot the same sunset for 12 hours at a time they can do so. If it needs to be cloudy, or sunny that can be accomplished by switching out the background and adjusting the light.
One limitation that shooting in this way still has is that the actors need to be about 15 to 20 feet away from the LED wall in order to create enough separation between the actors and background for the image to look realistic.
Apart from this one downside, this new technology of creative volume is a massive step forward in virtual set technology, which allows filmmakers a new degree of studio control and an ability for cinematographers to capture the images that they want in camera without leaving it up to post production.
Also remember this technology is still in its infancy. As it continues to get used on more shows in the future, such as the upcoming Netflix period production 1899, it will continue to improve, costs will slowly reduce and it will become more user friendly and faster for crews to work with.
We’re rapidly approaching the stage where filmmakers will be able to shoot scenes relatively easily in whatever photorealistic environments they imagine - without even needing a ton of post production manipulation.
As always technology pushes filmmaking forward, and will hopefully bring the industry back to the sweet spot of capturing films as much in camera as is possible.