encoding.info
You are hereSourcePredictionTransformEntropy codingFiltersFramesDecisionsRate controlPerceptionQualityBitstreamSettingsEncoder map
Page 1 of 4 in Pictures as numbers

Pixels, luma and chroma

Pictures as numbers · concept · added 2026-11-14

In Brief:
An encoder is usually handed three grids of integers: one of luma, a gray version of the picture, and two of color difference. It codes those numbers, and labels in the stream tell a display how to turn them back into colors.

In the common case an encoder is handed each frame as three grids of integers, called planes, and they are usually not red, green and blue. One is luma and two are color differences, called chroma. Luma is a weighted combination of the encoded red, green and blue values: a useful gray version of the picture, not a measurement of light or of perceived brightness. The encoder codes the three planes, and labels in the stream tell a display how to turn them back into colors.

Start with one pixel. Choose one below, or tap the cat or drag across it, to see a pixel's three numbers both ways.

The cat
32×32 pixels
Chosen pixel
Red, green, blue
Luma and chroma
Simplified demo: the cat is invented and is the picture this unit uses. Every bar's track runs from 0 to 255. A Cb or Cr bar grows from the tick at 128: to the right for more blue (Cb) or more red (Cr) than the luma, to the left for less. Values are full range and rounded to integers. With the cat in focus, the arrow keys move the chosen pixel.

From red, green and blue to luma and chroma

Luma is a weighted sum of red, green and blue: the encoded values a picture holds, not linear light. The two chroma values, Cb and Cr, say how far blue and red sit from luma. At 8 bits, 128 is the neutral value of each; when both are 128 the pixel is gray.

luma = 0.2126 × red + 0.7152 × green
       + 0.0722 × blue
Cb   = 128 + (blue − luma) / 1.8556
Cr   = 128 + (red − luma) / 1.5748

The weights and divisors are those of ITU-R BT.709, written here for values that use the full range from 0 to 255. BT.709's own nominal ranges at 8 bits are narrower: 16 to 235 for luma and 16 to 240 for chroma. The scaling changes; the idea does not.

In the figure at the top, choose Fur and then Sky. Both have a luma of 146. What tells them apart is chroma: Cb is 73 for the fur and 172 for the sky, and Cr is 179 and 93, on opposite sides of 128. A gray pixel, with red, green and blue equal, has both at exactly 128.

Take the planes apart

Choose how the cat is split into planes, then choose which planes to keep. A plane that is not kept is set to 128 everywhere, and the picture is rebuilt from the result. Each plane is drawn as a gray picture of its numbers; for Cb and Cr, mid-gray is 128.

Mean squared error is the average squared difference between the source and the rebuilt picture, over red, green and blue; zero means an exact match.

Source
Rebuilt
Mean squared error:
Simplified demo: the cat is invented. The conversion uses the formulas above with full-range values and rounds every plane to integers from 0 to 255. Rebuilt values outside 0 to 255 are limited to that range before the picture is drawn and the error is computed. A plane here is one component's grid of values; programs store the three in more than one layout.

For this cat:

What x264 is given

Converting changes the numbers. A label changes nothing; it tells whatever reads the stream how to interpret them. x264 has both. The x264 program converts its input to the format the encoder will use before the encoder sees it; the encoder is handed planes and codes them. --colormatrix converts nothing. It is parsed into a field that is written into the stream for a display to read, and the program's own conversion uses its converter's default weights whatever the option says. --range is both: a label in the stream and, when the input's range differs, a reason to convert. H.264 says these labels are not needed to construct the decoded samples; showing those samples with the right colors is another matter.

What to carry forward

Sources

Read 2026-11-14, at the commit linked.