The advent of digital cinematography has opened up new creative possibilities for how motion is captured. This article explores the influence of high frame rate (HFR) video playback, along with the associated motivations and controversy, with an eye for what this might hold for the future of cinema.
BACKGROUND
Although modern cinema uses a 24 fps time base, early film was projected with a wide variety of speeds. Prior to the 1930's, many silent films used just 15-20 fps, since this is when the illusion of continuous imagery begins. Then, with the advent of audio, frame rates were increased to the now-standard 24 fps, primarily because this was the minimum rate that would still produce acceptable audio when read from a 35 mm film reel.
(appears as a sequence of still frames)
(appears as stuttered video)
In any case, the overall strategy was to use as little film as possible. None of the motivations were to maximize the viewer's sense of realism footage was just deemed "good enough" without being prohibitively expensive. However, with digital capture, we're no longer bound by the same rules. Recent and upcoming productions are beginning to explore high frame rate (HFR) playback. HFR is already being used for sports and other HDTV broadcasts, and in cinema, Avatar 2 and The Hobbit are known productions targeting HFR release.
MOTIVATIONS
Even though about 15 fps is needed to initiate the illusion of continuous motion, the effect by no means stops there. Visual studies have shown that even if one cannot distinguish discrete images, a frame rate all the way up to 60-80 fps makes footage appear more lifelike by enhancing clarity and smoothness.
HFR also minimizes the appearance of motion artifacts especially when viewed in a theater. Moving objects may strobe or have a "picket fence" appearance as they traverse a large screen. At 24 fps, a 50 foot screen shows an object as jumping in 2 foot increments if that object takes one second to traverse the screen. This can appear as "judder" with fast panning and other types of camera movements.
Note: HFR is still cutting-edge on the web. For the smoothest playback, use a fast computer, don't move the mouse, and view multiple times to ensure it gets loaded into memory. Otherwise try downloading the videos here. Also, the above examples are not the result of a 3:2 pulldown from 60 fps to 24 fps; each was shot independently using the same shutter angle and pan rate.
Everything else being equal, one can also extract sharper and more precisely positioned stills with HFR. This is particularly helpful when a fashion video will also be used as a stills photo shoot, or when movie frames need to be pulled for print advertising. If HFR output isn't needed, at 48 fps one can always use frame skipping for backward compatibility with 24 fps.
Note: examples illustrate the effects while maintaining a standard 180° shutter angle. One could instead decrease the shutter angle at the same frame rate, but this would likely make the motion appear more stuttered.
HFR can also make the viewing experience more enjoyable by reducing eye-strain and fatigue. Since our visual system is designed to process continuous imagery, discrete footage can sometimes be tiring during extended viewing just as with flickering from CRT monitors or fluorescent lights. With 3D in particular, viewing fatigue is often cited as one of the biggest impediments to more widespread adoption.
Projectors also have the potential to brighten with HFR. With 24 fps, movie projectors typically show each frame 2-3 times for an overall refresh rate of 48-72 Hz. However, if those flashes are too bright, each frame will appear to flicker. With HFR, a frame is flashed fewer times permitting brighter projection without the associated flickering. This is especially important with 3D since traditional projection techniques are dimmer:
During intense or sudden action, our visual system often has a heightened sense of awareness, which can in turn slow our perception of time and increase visual input. Car chases, fight scenes, first-person perspectives and aerial footage are especially important. HFR can give these types of scenes an immediacy, fluidity and crispness not obtainable with standard frame rates:
Note: HFR is still cutting-edge on the web. For the smoothest playback, use a fast computer, don't move the mouse, and view multiple times to ensure it gets loaded into memory. Otherwise try downloading the videos here.
Note how the HFR example above appears more fluid particularly when the motorcycle becomes airborne. Such improvements are often dramatically better when viewed on a large theater screen.
The perceived difference also depends strongly on image content. In the upper motocross example, the difference is more pronounced due to the type of panning and amount of motion blur. Similarly, action, first-person and other dynamic shots stand to benefit more from HFR than do interviews and other static footage.
INITIAL REACTIONS
Despite its benefits, the HFR approach has not been without controversy. Over the past 80 years, viewers have grown to associate 24 fps with the familiarity and feel of traditional cinema. This typically includes its more pronounced motion blur and choppier cadence, along with the types of camera movements needed to avoid motion artifacts. Many have therefore begun using the term "hyper real" to refer to the new appearance of HFR footage. Such a reaction is understandable though, because after all, HFR is designed to look different.
However, some of the controversy has perhaps been misplaced, and may originate from technologies that don't use actual HFR footage. For example, modern televisions often have a feature that attempts to simulate HFR by adding artificial (interpolated) frames in between each actual frame:
(appears more natural)
(has ghosting and smearing artifacts)
Note: the above example is not intended to represent interpolated HFR with TV; it's depicted at 24 fps to clearly show the artifacts of interpolated frames, and how these frames don't appear as they would with standard video. Although interpolation artifacts persist at 60 fps and higher, they become much less distinguishable.
Unlike true HFR though, this approach doesn't affect motion blur, and can sometimes give footage a smeared appearance analogous to using a 360 degree shutter angle, or an early LCD screen with ghosting artifacts. Furthermore, HFR isn't as important on a home television as on the big screen.
The lure of HFR has also faced prior resistance, but for different reasons. Past attempts failed primarily due to cost, but this isn't a limiting factor nowadays; capable cameras are already available, modern projectors often just require a software update, and additional film isn't being consumed. The main consideration is visual effects (which can take twice the render time), but this will diminish as computer performance continues to improve.
DISCUSSION
When compact discs were first unveiled, the initial reaction by record connoisseurs was that the music had too much clarity and lacked the familiar and characteristic sound of a vinyl record. This closely echoes the early feedback about HFR. Similarly, while low frame rates will always have applications, having the creative flexibility to use other frame rates is virtually always beneficial. Even though a record can be mimicked with a compact disc recording, the opposite isn't always possible and the same can be said for low vs. high frame rates. Although not everything necessarily needs HFR, it may eventually become another creative tool, similar to how shutter angle is used currently.
Although great progress has been made with improving spatial resolution particularly with the advent of 4K cinema temporal resolution also deserves more exploration, and has a similar potential to enrich the cinematic experience. After all, real-life imagery is effectively received by our eyes as unlimited fps, infinite resolution 3D footage; it's our mind that processes this as a hybrid of video and motion-blurred stills. Higher frame rate, 4K+ footage gets us closer to that reality.