encoding.info
You are hereSourcePredictionTransformEntropy codingFiltersFramesDecisionsRate controlPerceptionQualityBitstreamSettingsEncoder map
Page 4 of 5 in Transform and quantization

Dead zone and rounding

Transform and quantization · concept · added 2026-11-13

In Brief:
The dead zone is the range of coefficient values that quantize to zero. Widening it removes more small coefficients, which can lower the cost of coding a block and increases the error.

A dead zone is the range of coefficient values that quantize to zero. Widening it removes more small coefficients, which can reduce the cost of coding a block but increases error. The default fixed-offset quantizers in x264 and x265 use a rounding offset that favors zero; it applies when trellis or rate-distortion optimized quantization is not used.

This simplified scalar model shows the idea. Here c is a transform coefficient and step is its quantization step size.

level = sign(c) × floor(|c| / step + offset)

floor rounds the magnitude down before the sign is put back, so the offset decides how far past a level a coefficient must be to reach it. In this model the decoder reconstructs a coefficient as level × step. It does not need to know the encoder's offset, which is not in the bitstream.

Why not round to nearest

With an offset of 0.5 the formula chooses the nearest level, and the zero band is already one step wide. A smaller offset moves each decision threshold farther from zero, so a coefficient needs a larger magnitude to reach the next nonzero level. The thresholds stay one step apart; only the central interval grows.

Round to nearest: offset 0.5 Wider zero band: offset 1/6 −3−2−10123
Dots are the values the decoder can output, in steps. Tick marks are where the encoder switches from one level to the next. The shaded band quantizes to zero.

Try changing the offset

The curve is an illustrative model of how coefficient values are spread: concentrated near zero, with larger magnitudes less common. Move the offset and the thresholds move. Everything between a pair of thresholds becomes the same level, and the shaded area is the share that becomes zero.

The bit figure treats every coefficient as independent and drawn from this one curve. It does not model coefficient order or block signaling, so it is not a bitrate.

0.17
← truncate toward zeroround to nearest →
−3−2−10123 Coefficient, in steps → ↑ Probability density
Zero-band width, in steps
Quantized to zero
Modeled bits per coefficient
Mean squared error
Simplified demo: coefficients follow a Laplace distribution with an average magnitude of one quantization step. The bit estimate is the entropy of individual quantized levels, assuming independent coefficients with this distribution. Error is in units of the step squared.

Reducing the offset from 0.50 to 0.17 raises the share of zeros from 39% to 56%. Modeled entropy falls by about 16%, while mean squared error roughly doubles. For this distribution, ordinary rounding with a step about 35% larger reaches the same modeled rate with about 13% less error:

QuantizerStepOffsetModeled bitsMean squared error
Wider zero band1.000.172.080.165
Round to nearest1.350.502.080.144

Real encoders code coefficient patterns using information such as position, neighboring coefficients and whether a block has any nonzero coefficients at all. Changing one coefficient to zero can therefore change the cost of signaling the rest of the block. This model leaves those effects out, so it cannot establish which offset works best in a real encoder.

What encoders use

EncoderSelected byNominal offset
x264Intra macroblock21/64 ≈ 0.328
x264Inter macroblock11/64 ≈ 0.172
x265I slice171/512 ≈ 0.334
x265P or B slice85/512 ≈ 0.166

x264 uses these in its fixed-offset path, when trellis does not run on the block. x265 uses them in scalar quantization with --rdoq-level 0.

x264 chooses per macroblock and x265 per slice, so in x265 an intra block inside a P or B slice gets the smaller offset. In x265 a later step, sign-bit hiding, can still change levels.

In x264 the luma offsets are settings: --deadzone-inter (default 21) and --deadzone-intra (default 11) give a nominal offset of (32 − value) / 64, so a larger value widens the zero band. x264 builds integer tables from that value, with rounding and a cap. Chroma stays at the defaults. --tune grain sets both to 6, among several other changes.

Trade-offs

Sources

Read 2026-11-13, at the commits linked.