encoding.info
You are hereSourcePredictionTransformEntropy codingFiltersFramesDecisionsRate controlPerceptionQualityBitstreamSettingsEncoder map
Page 3 of 6 in Perceptual optimization

Adaptive quantization in x264

Perceptual optimization · implementation · added 2026-11-13

x264's variance-based adaptive quantization (AQ) modes convert each macroblock's energy into a quantization parameter (QP) offset. Mode 1 uses a fixed logarithmic mapping. Modes 2 and 3 also use statistics of the whole frame. The formulas below follow the linked x264 source; the demonstration uses an invented distribution of block energies.

How energy is measured

ac_energy_mb runs on the source frame, once per 16×16 macroblock in the ordinary progressive case. It computes ssd − sum²/N, the sum of squared pixel values minus the squared pixel sum over the pixel count, for the luma block and for each chroma block, each against its own mean, and adds them. For 4:2:0 that is 256 + 64 + 64 = 384 samples.

The three curves

The plot shows the offset each mode gives a macroblock, against that macroblock's energy. The second slider moves the center of an invented distribution of block energies, on a log2 scale; modes 2 and 3 take their frame averages from that distribution. The strip of squares only illustrates rising contrast: its pixels are not used in the calculation. The readouts evaluate each mode at the center of the distribution. They are not the mean offset across the frame.

1.0
13
← lower-energy framehigher-energy frame →
Illustrative contrast strip: 20 × 3 macroblocks. More energy means more contrast inside the square.
+8 0 −8 ↑ QP offset log2(macroblock energy) → 4 9 14 19 central energy range
Mode 1: fixed referenceMode 2: auto-varianceMode 3: auto-variance, biased
At the center, mode 1
At the center, mode 2
At the center, mode 3
The formulas are x264's, at 8-bit. Simplified demo: the frame averages come from an invented distribution of macroblock energies, spread 1.5 either side of the slider value on the log2 axis (the shaded band). The noise strip illustrates the contrast; its pixels are not measured. Curves that leave the plot's vertical range are clipped. These are AQ offsets at this stage of the encoder, not final macroblock QPs.

The plot shows two things the mode names don't. Mode 1 has a fixed zero crossing, while the crossings of modes 2 and 3 move with the frame's energy distribution. And "auto-variance" does not force the frame's mean offset to zero: averaging the mode 2 formula over the frame cancels its first term and leaves strength × 0.5 × (mean(v²) − 14). Mode 3 adds a further energy-dependent bias, negative for low-energy blocks and positive for high-energy ones.

The formulas

Here v = (energy + 1)^0.125, avg is the frame mean of v, and mean(v²) is the frame mean of its square.

ModeQP offset
1strength × 1.0397 × (log2(max(energy, 1)) − 14.427)
2strength × (avg × (v − avg) + 0.5 × (mean(v²) − 14))
3Mode 2 plus strength × (1 − 14 / v²)

In mode 2 the first term separates blocks within the frame and the second is a frame-wide shift.

Sources

Read 2026-11-12 and confirmed on 2026-11-13 at the commit linked.