Adaptive quantization in x264
x264's variance-based adaptive quantization (AQ) modes convert each macroblock's energy into a quantization parameter (QP) offset. Mode 1 uses a fixed logarithmic mapping. Modes 2 and 3 also use statistics of the whole frame. The formulas below follow the linked x264 source; the demonstration uses an invented distribution of block energies.
How energy is measured
ac_energy_mb runs on the source frame, once per 16×16 macroblock in the ordinary progressive case. It computes ssd − sum²/N, the sum of squared pixel values minus the squared pixel sum over the pixel count, for the luma block and for each chroma block, each against its own mean, and adds them. For 4:2:0 that is 256 + 64 + 64 = 384 samples.
The three curves
The plot shows the offset each mode gives a macroblock, against that macroblock's energy. The second slider moves the center of an invented distribution of block energies, on a log2 scale; modes 2 and 3 take their frame averages from that distribution. The strip of squares only illustrates rising contrast: its pixels are not used in the calculation. The readouts evaluate each mode at the center of the distribution. They are not the mean offset across the frame.
The plot shows two things the mode names don't. Mode 1 has a fixed zero crossing, while the crossings of modes 2 and 3 move with the frame's energy distribution. And "auto-variance" does not force the frame's mean offset to zero: averaging the mode 2 formula over the frame cancels its first term and leaves strength × 0.5 × (mean(v²) − 14). Mode 3 adds a further energy-dependent bias, negative for low-energy blocks and positive for high-energy ones.
The formulas
Here v = (energy + 1)^0.125, avg is the frame mean of v, and mean(v²) is the frame mean of its square.
| Mode | QP offset |
|---|---|
1 | strength × 1.0397 × (log2(max(energy, 1)) − 14.427) |
2 | strength × (avg × (v − avg) + 0.5 × (mean(v²) − 14)) |
3 | Mode 2 plus strength × (1 − 14 / v²) |
In mode 2 the first term separates blocks within the frame and the second is a frame-wide shift.
- The zero point of mode 1. 214.427 is about 22,000, a root-mean-square (RMS) deviation near 7.6 code values over 384 samples. The source says the constants were chosen to give about the same overall bitrate as without AQ.
- Bit depth. The mode 1 constant rises by 2 per extra bit (18.427 at 10-bit). Modes 2 and 3 scale energy down instead.
- Mode 3's bias keys on energy, not brightness. The extra term is negative below an energy of about 38,000 and positive above it. x264's help text calls it "bias to dark scenes", but luma level never enters the formula.
- Defaults. Mode 1 at strength 1.0, in every preset except ultrafast, which turns AQ off.
--tune psnrturns AQ off;--tune ssimselects mode 2.
Sources
Read 2026-11-12 and confirmed on 2026-11-13 at the commit linked.
- x264 at commit c24e06c (GitHub mirror):
ac_energy_mb,x264_adaptive_quant_frame, the mode 1 formula and its floor at 1, the--aq-modehelp text, the defaults,--tune psnr,--tune ssim. - Not from x264: the distribution of block energies and the noise strip in the demo, which are invented. The interlaced paths were not read.