encoding.info
You are hereSourcePredictionTransformEntropy codingFiltersFramesDecisionsRate controlPerceptionQualityBitstreamSettingsEncoder map
Page 3 of 4 in Pictures as numbers

Bit depth and banding

Pictures as numbers · concept · added 2026-11-14

In Brief:
Bit depth sets how many levels a sample can take. With too few, a smooth gradient turns into visible bands. More bits make the steps smaller, and dither trades the bands for noise.

Bit depth sets how many integer codes a sample can take: 8 bits allow 256 and 10 bits allow 1,024. Rounding a smooth gradient to a limited set of codes creates steps. When the steps are visible as contours, that is banding. More levels make the steps smaller, and dither can make contours less apparent by replacing them with variation between pixels.

Count the levels

The strip is a piece of sky 320 pixels wide whose brightness rises smoothly from 50% to 56% of full scale. Bits sets how many bits each sample gets:

level = round(value × (2bits − 1))
stored value = level / (2bits − 1)

At 8 bits full scale is 255, so the strip runs from 127.5 to 142.8 and has only the whole numbers from 128 to 143 to land on. Dither adds a random amount of up to half a level to each pixel before rounding, so pixels near a step fall on either side of it.

Each setting starts again from the smooth source. Writing an already rounded 8-bit picture with 10 bits cannot bring back the values in between: two pixels that both rounded to 135 are still equal.

Both pictures use the same eightfold contrast boost around the strip's middle value, so that small differences are easy to compare. The pictures exaggerate the effect; the plot and the readouts use the unboosted values, in 8-bit levels. Widest band is the longest run of pixels in the middle row that share a level. Pixel error is the mean squared difference from the source, in squared 8-bit levels. Column-average error first averages the 48 samples in each column, then compares that average with the source: it shows how well the strip represents the gradient when those samples are combined, and it does not simulate a viewer. With Dither on, watch the errors; dither makes short runs of its own by chance, so Widest band says little.

8
← 5: coarse10: fine →
Source, contrast ×8
Stored, contrast ×8
Value, in 8-bit levels Pixel
SourceStored, middle row
Levels used
Widest band, in pixels
Pixel error
Column-average error
Simplified demo: the strip and the dither are invented. The source is held at full precision; a real source already has a bit depth of its own. The dither is independent random noise, the same pattern at every setting. Levels used and the two errors are counted over all 48 rows; the plot and Widest band use the middle row. There is no encoder here: this is storage alone. The pictures are drawn as 8-bit grays; they are not a test of what a 10-bit screen shows.

For this strip:

Bit depth in x264 and x265

An encoder can add rounding error of its own even when its samples have many levels: quantization can round away the small residual a gradient leaves. This demo isolates the sample grid. It has no quantizer and does not predict banding in an encoded video.

In x264 --output-depth sets the depth the encoder works in, 8 or 10, and the depth goes into the stream. An ordinary lossy 10-bit 4:2:0 encode gets the High 10 profile, which a decoder has to support. When the input has more bits than the output, the program's depth filter dithers on the way down. Its dither is error diffusion, not the demo's independent noise.

The quantization parameter range also grows with bit depth; Quantization and the quantization parameter has the numbers.

In x265 --output-depth has to match the depth the library was built for, and --dither, off unless given, dithers an input that has more bits than that.

Trade-offs

Sources

Read 2026-11-14, at the commits linked.