Video compression is commonly thought of as being unique to digital, but it's also been around since the early days of analog. Compression has just become more sophisticated since then. In this article, we will take a look back at one early strategy in particular: 4:2:2, 4:1:1 and 4:2:0 chroma subsampling.
STRATEGY
Vision specialists have long known that our eyes are much less sensitive to color resolution than we are to brightness resolution:
This became a central motivation behind early forms of analog and digital compression. Video signals would be separated into a lightness or "luma" component and two color or "chroma" components, similar to how images can be separated into three red, green and blue (RGB) components. The luma and chroma components would then be referred to as YUV (with analog) or YCbCr (with digital) as opposed to RGB.
Once separated, the chroma resolution would then be reduced by half or more through a process called "chroma subsampling." The end result is a video signal that appears more detailed at the same broadcast bandwidth, since the luma component occupies a greater fraction of the video signal:
(Original Image)
(As Recorded on DVD and Blu-ray)
The technique is also relatively simple to implement, and does not require fast processors to decode. For these reasons the strategy has been highly successful and persisted to this day.
HOW IT WORKS
Early monitors showed images by progressively scanning across each horizontal line of pixels, usually from top to bottom in rapid succession. As each line was scanned, values would be sent less frequently for chroma than for luma.
Although modern display devices don't all work this way, the concept of scanlines is still important because chroma subsampling types are specified horizontally. Each type is often listed as a ratio between the rate at which luma and chroma values are sent as a line is scanned. This ratio is typically based on four luma values, and takes the form 4:X:Y, where X and Y are the relative number of chroma values for in rows of a conceptual 4x2 pixel block. The example below illustrates how these ratios affect resolution for a simple 4x2 pixel image:
Using the standard nomenclature, 4:2:2 means each horizontal scanline has 2 chroma values for every 4 luma values. Similarly, 4:1:1 means 1 chroma value is sent for every 4 luma values sent, and 4:4:4 means no chroma subsampling. However, this is not entirely consistent. 4:2:0 would imply that for every 4 luma values, there would be 2 for the first chroma component and 0 for the second but this wouldn't produce full color images. In practice, 4:2:0 instead means there are two of each chroma sample per scanline, but that these are only present every other line.
COMPRESSION ARTIFACTS
Since chroma subsampling effectively decreases color resolution, it will be most visible near the edges of sharp color transitions. The example below shows what this looks like with a simple 8x8 pixel image:
The net effect is usually a decrease in color saturation within fine details. While this typically doesn't decrease saturation within large-scale objects, it can still be visible if these objects contain fine color patterns. The colorful ceiling pattern below is particularly susceptible:
However, the strength of this effect depends heavily on the type of detail. The next example includes closely-spaced but similarly-colored roof tiles, and depicts a subtler difference:
With each setting above, pay particular attention to the red rim across the top roof, the vertical red chimneys, the red window trim and the diagonal tiles on both the upper and lower rooftops. The most visible artifact is likely lower color saturation within these regions. Also note how the various settings affect vertical versus horizontal color details differently.
DISCUSSION
Although chroma subsampling has been an easy yet effective compression technique since the early days of video, it can create noticeable artifacts. Digital techniques have also become far more sophisticated since then. Whereas subsampling utilizes a simple image-wide reduction in color resolution, modern digital codecs can analyze image content and decide how to prioritize that detail. Regions with lower brightness, less saturation and coarser detail can all be treated differently with digital, for example.
Consumers have also become more discerning, and have grown to expect more when it comes to image quality. Moving beyond the traditional 4:2:0 compression used with DVD and Blu-ray has therefore been a recent trend. Ultimately, feeding full 4:4:4 data into a modern digital codec makes the most of that imagery and has the potential to noticeably improve image quality.
FURTHER READING
- Note that the 4:X:Y terminology only applies to codecs used after imagery has been captured not to the camera sensors themselves. See the article on Bayer sensor strategy for more.
- To understand the limits of luma resolution, see the tutorial on resolution versus aliasing.