Bit depth and banding
Bit depth sets how many integer codes a sample can take: 8 bits allow 256 and 10 bits allow 1,024. Rounding a smooth gradient to a limited set of codes creates steps. When the steps are visible as contours, that is banding. More levels make the steps smaller, and dither can make contours less apparent by replacing them with variation between pixels.
Count the levels
The strip is a piece of sky 320 pixels wide whose brightness rises smoothly from 50% to 56% of full scale. Bits sets how many bits each sample gets:
level = round(value × (2bits − 1)) stored value = level / (2bits − 1)
At 8 bits full scale is 255, so the strip runs from 127.5 to 142.8 and has only the whole numbers from 128 to 143 to land on. Dither adds a random amount of up to half a level to each pixel before rounding, so pixels near a step fall on either side of it.
Each setting starts again from the smooth source. Writing an already rounded 8-bit picture with 10 bits cannot bring back the values in between: two pixels that both rounded to 135 are still equal.
Both pictures use the same eightfold contrast boost around the strip's middle value, so that small differences are easy to compare. The pictures exaggerate the effect; the plot and the readouts use the unboosted values, in 8-bit levels. Widest band is the longest run of pixels in the middle row that share a level. Pixel error is the mean squared difference from the source, in squared 8-bit levels. Column-average error first averages the 48 samples in each column, then compares that average with the source: it shows how well the strip represents the gradient when those samples are combined, and it does not simulate a viewer. With Dither on, watch the errors; dither makes short runs of its own by chance, so Widest band says little.
For this strip:
- With Dither off, 8 bits put it on 16 levels. The widest band is 21 pixels and the pixel error is 0.084.
- At 10 bits, still with Dither off, it lands on 62. The widest band is 6 pixels and the pixel error is 0.005. The steps are about a quarter the height, not gone.
- Dither raises the pixel error and lowers the error of the averages. At 8 bits with Dither on, the pixel error doubles, from 0.084 to 0.169, and the column-average error falls from 0.084 to 0.003. The staircase becomes noise around the true line.
- With Dither back off, 5 bits round the strip to 2 levels. One step, near the middle. Five bits have 32 codes in all; this narrow strip reaches two of them.
Bit depth in x264 and x265
An encoder can add rounding error of its own even when its samples have many levels: quantization can round away the small residual a gradient leaves. This demo isolates the sample grid. It has no quantizer and does not predict banding in an encoded video.
In x264 --output-depth sets the depth the encoder works in, 8 or 10, and the depth goes into the stream. An ordinary lossy 10-bit 4:2:0 encode gets the High 10 profile, which a decoder has to support. When the input has more bits than the output, the program's depth filter dithers on the way down. Its dither is error diffusion, not the demo's independent noise.
The quantization parameter range also grows with bit depth; Quantization and the quantization parameter has the numbers.
In x265 --output-depth has to match the depth the library was built for, and --dither, off unless given, dithers an input that has more bits than that.
Trade-offs
- More bits shrink the steps; they do not remove them. From 8 to 10 bits this strip's widest band goes from 21 pixels to 6.
- Dither replaces bands with noise. The pixel error goes up, and the noise is detail an encoder then has to spend bits on or discard. This page does not measure that cost.
- A 10-bit stream needs a decoder that supports its profile. For ordinary lossy 4:2:0 that is High 10 in H.264 and Main 10 in High Efficiency Video Coding.
- Sample size is not stream size. Ten bits per sample is 25% more than eight if tightly packed. Raw storage also depends on packing and padding, and compressed size depends on the encoding.
Sources
Read 2026-11-14, at the commits linked.
- x264 at commit c24e06c (GitHub mirror): the
--output-depthoption and the two depths the program handles inx264.c; the default depth incommon/base.c; the High 10 choice and the depth written to the sequence header inencoder/set.c; the dither, the shift used when raising depth and the choice between them infilters/video/depth.c. - x265 at commit 4191822 (GitHub mirror): the compiled-depth check in
source/common/param.cpp; the dither default insource/x265cli.hand its use insource/abrEncApp.cpp. - Not checked: the dithering function x265 calls; which depths a given build of either encoder includes; whether 10-bit encoding of an 8-bit source changes banding after quantization, which is often claimed and is not measured here; the specifications' text on bit depth; libaom and SVT-AV1; which devices decode High 10 or Main 10.