Pixels, luma and chroma
In the common case an encoder is handed each frame as three grids of integers, called planes, and they are usually not red, green and blue. One is luma and two are color differences, called chroma. Luma is a weighted combination of the encoded red, green and blue values: a useful gray version of the picture, not a measurement of light or of perceived brightness. The encoder codes the three planes, and labels in the stream tell a display how to turn them back into colors.
Start with one pixel. Choose one below, or tap the cat or drag across it, to see a pixel's three numbers both ways.
From red, green and blue to luma and chroma
Luma is a weighted sum of red, green and blue: the encoded values a picture holds, not linear light. The two chroma values, Cb and Cr, say how far blue and red sit from luma. At 8 bits, 128 is the neutral value of each; when both are 128 the pixel is gray.
luma = 0.2126 × red + 0.7152 × green
+ 0.0722 × blue
Cb = 128 + (blue − luma) / 1.8556
Cr = 128 + (red − luma) / 1.5748
The weights and divisors are those of ITU-R BT.709, written here for values that use the full range from 0 to 255. BT.709's own nominal ranges at 8 bits are narrower: 16 to 235 for luma and 16 to 240 for chroma. The scaling changes; the idea does not.
In the figure at the top, choose Fur and then Sky. Both have a luma of 146. What tells them apart is chroma: Cb is 73 for the fur and 172 for the sky, and Cr is 179 and 93, on opposite sides of 128. A gray pixel, with red, green and blue equal, has both at exactly 128.
Take the planes apart
Choose how the cat is split into planes, then choose which planes to keep. A plane that is not kept is set to 128 everywhere, and the picture is rebuilt from the result. Each plane is drawn as a gray picture of its numbers; for Cb and Cr, mid-gray is 128.
Mean squared error is the average squared difference between the source and the rebuilt picture, over red, green and blue; zero means an exact match.
For this cat:
- Luma alone leaves a recognizable gray cat. Much of its structure remains, but color differences also define edges: the fur and sky pixels of the first figure both become gray 146.
- Keeping Cb and Cr sets every luma value to 128. The color-difference patterns remain and the original shading is gone. 20 of the 1,024 rebuilt pixels fall outside 0 to 255 and are limited.
- Squared error is lower for Cb and Cr than for Luma only: 2,388 against 2,745. It counts numerical differences in red, green and blue, not whether the cat is recognizable or its colors look right. Compare the two pictures yourself.
- In red, green and blue every plane looks like the cat. Keeping one of them leaves an error of 3,694, 4,262 or 2,403, and wrong colors everywhere.
- The round trip is not exact. With all three luma and chroma planes kept the error is 0.27, and no value is off by more than 1: rounding each plane to an integer costs that. With red, green and blue it is 0.
What x264 is given
Converting changes the numbers. A label changes nothing; it tells whatever reads the stream how to interpret them. x264 has both. The x264 program converts its input to the format the encoder will use before the encoder sees it; the encoder is handed planes and codes them. --colormatrix converts nothing. It is parsed into a field that is written into the stream for a display to read, and the program's own conversion uses its converter's default weights whatever the option says. --range is both: a label in the stream and, when the input's range differs, a reason to convert. H.264 says these labels are not needed to construct the decoded samples; showing those samples with the right colors is another matter.
What to carry forward
- The encoder codes three planes of integers. Colors are what a display makes of them.
- Luma is a gray version of the picture; Cb and Cr are two maps of color difference. At 8 bits, a pixel with both at 128 is gray.
- A color label describes the planes; it does not change them. A wrong
--colormatrixleaves the coded values as they were and can make a display show shifted colors.
Sources
Read 2026-11-14, at the commit linked.
- x264 at commit c24e06c (GitHub mirror):
init_vid_filtersinx264.c, which chains the format conversion and the depth filter in front of the encoder; thecolormatrixoption incommon/base.c; the field copied into the sequence header and the range flag and matrix value written inencoder/set.c; the range comparison and the converter's weights infilters/video/resize.c. - ITU-T Recommendation H.264, the 05/2003 edition, Annex E clause E.2.1 only, read as extracted text: equations E-1 to E-9 and Table E-5 give the weights (0.2126 and 0.0722 for ITU-R BT.709), the full and narrow ranges, and the statement that these parameters are not required for constructing the samples in the decoding process. Later editions may number these differently.
- ITU-R Recommendation BT.709-6 (06/2015), items 3.1 to 3.4 and 4.6 only, read as extracted text: red, green and blue are pre-corrected, not linear, before the weighted sum, which BT.709 calls the luminance signal; the divisors 1.8556 and 1.5748; and the 8-bit levels 16, 235 and 240 (64, 940 and 960 at 10 bits).
- Not checked: the rest of BT.709, and BT.601; the transfer characteristics that turn light into red, green and blue values; whether any part of x264 outside the files listed reads the matrix field; x265, libaom and SVT-AV1; what displays do with a missing or wrong label.