Shooting VFX Elements
Shooting VFX elements is an important, often crucial, part of the process. It doesn’t take more than small mistake, a detail that goes unnoticed, or disregard to a seemingly unimportant factor to turn a much needed element into a completely unusable piece of footage. Often, a quick adjustment or an extra half hour to set things properly is all that’s needed. But the reality is that VFX elements shoots are often
A page from a VFX plates list, Boardwalk Empire.
Boardwalk Empire
© Home Box Office (HBO), courtesy of Brainstorm Digital.
pushed to the back end of the day’s schedule, and when the clock is ticking and the crew is tired, things tend to get rushed. Understandably, getting the actual shots and the lead performances is always first priority. But it is important to remember that those VFX elements ARE a part of the shot, sometimes an indispensable part. The guidelines and suggestions that follow are meant to help you get both the main action and the VFX elements on schedule, and ensure that the elements are shot in an optimal way.
To be clear, a VFX element can be anything that is captured separately and used in another shot (the element often needs to be separated from the background via roto or green screen). Elements can be a single person or a group, crowds, objects, specials effects such as explosions, smoke or blood squibs, backgrounds, environments, skies, etc. Regardless of what the element is, the importance is to know where and how the element will be used. VFX elements can be generally divided into two categories: shot-specific elements; and generic elements. The first category includes elements that will be used in a particular shot, which mandates a strong coherence in lighting, camera angle, and lens type between the original “master” shot and its accompanying VFX element/s. Crowd tiling is a good example (and will be discussed separately later in the chapter). Generic elements on the other hand are intended to be used in any number of different shots (or be part of a library of elements), and therefore must be shot in a way that makes them usable in a variety of scenarios. As I go through the guidelines, I will specify the difference in approaches for each category.
Camera Movement
No human can repeat a camera move precisely. Highly skilled camera operators and dolly grips may get a pretty close match, but pretty close is just not enough for VFX. The slightest mismatch, slip or slide in camera movement is immediately noticeable and is an absolute shot killer. VFX elements must be precisely tracked to the shot, and nothing should be floating around loosely. It is therefore absolutely crucial to shoot VFX elements with a fully locked off camera, so that they can later be tracked to the master shot by the VFX team. The camera should be mounted on a sturdy base to prevent any vibrations or accidental movement. It is true that footage can be stabilized in post to a certain extent, but the motion blur, perspective shift and parallax that result from a jittery camera cannot be undone.
A hand-held camera is therefore not an option for shooting VFX elements. This rule applies equally to generic and shot-specific elements. The only exception is when a motion-controlled camera is used.
High quality motion control rigs let you program a move and repeat it over and over with full accuracy.
They are expensive and take the extra time to set up, but they are indispensable for complex VFX shots that involve wide, sweeping camera moves and require several repeated shooting passes for different elements (like the Ellis Island example in the previous chapter). The more advanced motion control systems can also output exact motion data that saves a lot of time and money on camera tracking. That said, for any other manually controlled camera, the rule for shooting VFX elements is simple: keep the camera static.
Camera Angle and Position
If the elements are shot-specific, it’s very important to match the master shot’s camera angle as closely as possible. For example, if the master plate was shot from a high angle and tilted down, the elements will not work in terms of perspective if they are shot from a straight or low angle. For this reason, you should always get the master shot first before shooting elements for it. The distance of the object from the camera is important mainly when using a wide lens, because the perspective distortion on objects that are close to the lens is much stronger than for distant ones (more on this in the next section).
Matching the camera angle for the element is easy if the master shot was a locked-off, and can usually be
“eyeballed” by the camera operator simply by looking at a single frame of the master shot. However, if
Replacing the NYC house with a Victorian London house on The Wolf of Wall Street. Our VFX work on this shot went very smoothly, mainly because the movie’s second unit made sure to match the original camera’s lens, position, and angle precisely when they shot the London plate.
The Wolf of Wall Street © Paramount Pictures, Red Granite Pictures, Appian Way, Sikelia Productions, EMJAG Productions. Visual effects by Brainstorm Digital.
the camera was moving, the camera operator needs to pick one position/angle along the move and lock the camera to that position for the elements shoot. Usually, it’s best to choose a position/angle that’s roughly mid-way through the move, to minimize the amount of stretching on the element when it is projected on a card in 3D space.
Generic elements, on the other hand, should always be shot at a straight angle. It is much easier for the VFX team to add some fake perspective skew to a 2D element than try to undo one that is already baked in. Generic elements that are shot from an extreme low or high angle will be of very limited use. As for position, it’s best to try to get as close as possible to maximize resolution and the amount of detail, as long as a medium lens is used and the element does not break frame (see below).
Lens Type
Ideally, shot-specific elements should be captured with the same lens that was used in the master shot.
However, this only works when the angle and the distance are also matched. For example, a wide lens will characteristically produce more perspective distortion on objects that are close to the lens, while objects in the distance will be much less distorted. So, if you are generating elements that are intended
For this element, a wide lens and a low angle were needed to match the shot. A setup such as this, however, is not really suitable for generic elements—the wide lens exaggerates perspective and the low angle will not work for most shots.
Sons of Liberty © Stephen David Entertainment, History Channel, A+E Studios. Courtesy of Brainstorm Digital.
to be composited in the distance, but actually shoot them much closer to the camera, you will be adding unwanted distortion. This will make it hard (or impossible) for the VFX team to get the elements to look right in the shot, since perspective distortion cannot be easily undone. Bottom line: if you can place the objects where they need to be, then go ahead and match the lens to the master shot. But if you can’t match the distance (because of physical set limitations or because you want to capture the elements at full resolution), then it is best to use a medium lens, which will minimize perspective distortion regardless of the distance.
As for generic elements, wide angle lenses should be avoided, because the perspective distortion drastically reduces the usability of the elements. As I said before, “faking” perspective distortion is easier than removing it. 35–65mm lenses are best for generic elements, as they are neither too wide nor too long.
Framing
An element that breaks frame is no longer usable. Even if most of it is in frame, or it’s in frame most of the time, the part or the moment when it breaks frame is still unusable. Filmmakers often ignore a little frame-breaking, assuming that having “most” of the element in frame is good enough, and that the VFX team will somehow recreate the missing parts. An element that breaks frame is sharply cut off at the frame edge. Think of it: if the element is an explosion for example, the VFX team will need to use additional elements and seamlessly blend them in to extend the missing pieces, or even trash the element footage and go for a full-on CG simulation. Imagine what it will take to extend or recreate an actor who goes, partially or fully out of frame. Whether it needs to be extended in 2D or 3D, the outcome is similar: an element that was originally shot to save time and money will now cost a lot more just to be “fixed,” when in fact this could have been prevented by simply moving the camera back a bit or instructing the actor not to step beyond a certain boundary.
For generic elements, it is of course crucial to keep everything in frame. This is easy when the element is a defined entity such as a person or an object, and a bit more challenging with amorphous and unpredictable elements like smoke, splashes, charges, or debris. Large explosions are especially hard to frame because they are usually a one-off event that can’t be rehearsed. When in doubt, remember that it’s usually preferable to have the element smaller but within frame than larger but breaking frame.
When trying to keep an element in frame, it is preferable by far to move the camera back rather than switch to a wider lens. (As already discussed, you really want to avoid the perspective distortion of wide lenses.) Shot-specific elements are somewhat different—it makes sense that if the angle, distance, and lens are matched to the master shot, then elements can naturally break frame as they would do in the master shot. Still, I always recommend pulling the camera back slightly to add a bit of a buffer area in frame. The VFX team can always push back in, and having a few more pixels on the edges is always a good safety measure.
Choosing the Right Background
Since in most cases the element will be separated from the background, it’s important to make sure that the background allows for successful extraction or roto work. I am often asked on set about the preferred type of background for a specific element. Should it be a green screen? Black screen? Or maybe a plain gray or white background?
Let’s first rule out white—it’s a terrible choice for a background. Having no saturated hue, it cannot work as an extractable color like green, and in addition it will brighten up any edge that’s not 100% solid, making it very hard to comp the element over any background but the brightest. In short, do not use a white background.
Gray, like white, is not extractable, but at least will not brighten the edges and will work better over a variety of comped backgrounds. If a green screen is not available and roto is the only option, a solid gray cement wall or a gray panel will at least provide consistent background without too much contrast or saturated colors.
A green screen (or blue screen if the element has green in it) should be used for most types of elements.
It is the best choice for extraction and will usually turn out to be the most economical choice down the line, provided it is properly set up. (In the next section I will discuss setting up green screens in detail.) A black screen is a preferable background for elements that are extracted using luminance rather than hue. This includes self-illuminated elements like fire, flames, muzzle flashes, lights, flares and sparks, as well as bright white elements such as steam, white smoke, snow, and rain. For all these elements, a black screen provides more contrast between foreground and background than a green screen, and preserves the original color of the element, since no spill or green bleed is present.
There are certain circumstances when no screen can be used as a backdrop—for example, when shooting explosions that are too large to be covered by a screen. In such cases, it is always better to shoot against This muzzle flash/smoke element looks great on the first frames, but a few frames later the smoke shoots forward and breaks frame right, making it quite unusable as a generic element, except for extreme close-up situations. If on the first take the element breaks frame, the best thing to do is push the camera back. It is usually better to have the element smaller in frame than to have it break frame.
Sons of Liberty © Stephen David Entertainment, History Channel, A+E Studios. Courtesy of Brainstorm Digital.
a relatively consistent background. Overcast or uniformly blue sky are preferable to partially cloudy skies.
A clean concrete wall is better than a brick wall. Luminous elements like explosions can also be shot against a dark night sky as a natural replacement for a black screen.
Frame Rate
There are advantages in shooting elements at a higher frame rate. Special effects that need to appear much larger than their actual size (a giant water splash or a big explosion) are usually timed slower to feel bigger—this is a venerable well-known trick for miniature work. Shooting an element at 48fps means that the VFX artists can smoothly play it two times slower, without the degrading artifacts of artificial retime.
Double, triple, or even quadruple frame rates are quite feasible with digital cameras, so it’s becoming common practice to shoot all generic elements at higher frame rates. It’s probably worth it—even if the element will not be slowed down, it doesn’t hurt to have that option in store. The only difference will be a certain lack of motion blur on fast-moving elements. This is hardly an issue since motion blur can be added back when the slow-motion element is retimed back to actual speed.
When it comes to shot-specific elements though, shooting at a different frame rate than the master shot does not make much sense (unless of course, the idea is to have that element move in a different speed than the rest of the shot, as part of the effect). Normally, you’d want to keep everything looking the same and seamlessly integrated, which means shooting the element at the same frame rate as the master shot.
Green Screens
In Chapter 4 I discussed green screens from the compositing stand point—the challenges of extraction, soft edges and integrating green screen footage with a different background. But the success of a green screen shot undeniably starts on the set. A bad green screen setup can easily transform a relatively The explosion element used here was too big to be shot against a green screen. Instead, it was shot against a clear sky, a “poor man’s blue screen” of sorts.
Boardwalk Empire © Home Box Office (HBO). Visual effects by Brainstorm Digital.
straightforward VFX shot into a nightmare. What was supposed to be a day worth of compositing may turn into a week of struggle. The truth is that it does not take a whole lot of work or expertise to set up a proper green screen shot—often the setup is botched by simple lack of attention or unintentional neglect rather than difficult conditions or lack of resources. In other words, an extra ten minutes of straightening some folds or slightly rotating the screen can save hours, if not days, of VFX work down the line.
Action Coverage
“Keep it away and keep it inside” is the motto here. This refers to the action, not the screen. Actors or other subjects should never stand too close to the green screen. A proximity of less than 5 to 7 feet (depending on the lighting and the size of the screen) puts the actor into the “death zone” of extreme spill and light bounce. As I said in Chapter 4, small or moderate amounts of spill can be corrected in comp, but there’s a certain point where excessive spill and light bounce affects the color, contrast, and overall look of the subject in a way that is very hard to correct. Additionally, when a subject is bathed in heavy green spill, it might be altogether impossible to extract, as the extraction algorithm cannot make a clear distinction between the subject and the screen. Because the green screen is bright and saturated, it acts as a strong light reflector, creating an unnatural backlit edge on the subjects, so keep the action a safe distance from the screen.
Consequently, “keep it inside” means that you should always use a screen that is large enough to cover all the action in the designated area. By “designated area” I mean the specific area that will need to be replaced or treated by VFX. For example, if the shot is a wide street view, and only one façade needs to be replaced, only that façade needs to be covered by green. On the other hand, if the entire background needs to be replaced, then the screen must cover the entire action area.
It is not unusual that action goes beyond the edge of the screen. This can happen because the size of the screen was miscalculated, a smaller screen was used due to rigging limitations, or a suitably sized screen was simply not available. It can also happen because the actors are not aware of the perimeter of the safe action area. There’s a common assumption that it’s good enough to have “most” of the action within the screen, and that a little bit of overflow is negligible. But as I explained in Chapter 4, roto is much more limited than extraction when it comes to wispy or semi-transparent edges like hair, or areas of motion blur or defocus. Because of that, there can be a very noticeable drop in extraction quality between the areas that were on the green screen and those that spilled out.
Green screens come in various sizes, so planning is key here. It’s a fine balance between not going overboard (large screens take more time to set up and require more rigging for stability and safety) and making sure the action is fully covered. A minimum distance from the screen should also be factored in.
The farther away the screen is from the subject, the larger it needs to be. It’s also important to remember that the screen only needs to cover the action and areas we want to keep. Anything else can be simply discarded and replaced. So if, for example, the action area is just a quarter of the frame, and the rest does not require any separation, there’s absolutely no need to set up a giant screen to cover the entire frame.