encoding.info
You are hereSourcePredictionTransformEntropy codingFiltersFramesDecisionsRate controlPerceptionQualityBitstreamSettingsEncoder map
Page 2 of 6 in Perceptual optimization

Adaptive quantization

Perceptual optimization · concept · added 2026-11-12

In Brief:
Adaptive quantization changes the quantization parameter block by block. The common kind quantizes smooth regions more finely, where errors can be easier to see, and textured regions more coarsely, aiming at better perceived quality for the bits available.

Adaptive quantization (AQ) adjusts the quantization parameter (QP) block by block across a picture. The common variance-based kind gives smooth regions finer quantization, because errors can be easier to see there, and allows coarser quantization in textured regions, where some errors are less noticeable. The aim is better perceived quality for the bits available.

Why one QP per frame falls short

Using the same QP throughout a frame treats smooth and textured regions alike. But the same-sized error can be more noticeable in a smooth region than in texture.

Smooth block Textured block Errors stand out against a smooth background Texture can make errors less noticeable Original Decoded
Conceptual illustration. The error amplitude is identical on both sides.

In a smooth region the error can show as banding or blocking; in grass or foliage it can be much harder to see. This is spatial masking, and variance-based AQ uses it when choosing local QP offsets.

Try changing the strength

Sixty blocks of a frame: sky on top, a tree and grass below. Each block gets a QP offset from its activity score. Move the slider: at 0 every block has the base QP; as strength rises, QP falls in the sky and rises in the grass.

This illustration holds the base QP at 28. Both the QP offsets and the bit estimates use simplified formulas. It shows how stronger AQ can shift the share of bits toward smooth regions; it does not simulate rate control or predict an encoder's bitrate.

1.0
← off: one QP for every blockstrong →
Illustrative block activity
Sky on top, tree and grass below. Darker means more activity.
QP per block (base QP 28)
lower QP, more bits   higher QP, fewer bits
QP range
Modeled bit share, smooth blocks
Modeled bit share, textured blocks
Simplified demo: 60 blocks with assigned activity scores, not variance measured from pixels. The offset is a straight line in that score, which is not the logarithmic mapping described below, and the bit estimate is invented. The percentages are shares of the model's own total, which also changes with strength; they are not encoder measurements.

Equal QP does not mean equal bits: at strength 0 the 31 textured blocks already take 78% of the modeled bits. At strength 1 the split is 42% smooth to 58% textured, with QP running from 23 to 34.

In a real encoder

  1. Measure each block's activity before encoding, usually its block energy: how far the pixels stray from the block's mean.
  2. Convert it to a QP offset. In these variance-based modes, lower-activity blocks generally get lower offsets than higher-activity ones. The mode sets the reference point and the size of the difference; offsets are not necessarily centered on zero.
  3. Rate control responds. AQ changes how bits are spread across the picture. When a bitrate target is set, rate control adjusts quantization to work toward it. With quality-based encoding such as constant rate factor, changing AQ can also change the total bitrate.
  4. The bitstream carries the result. The decoder gets what it needs to work out each region's quantizer and does not repeat the activity analysis. In H.264 that is a QP delta per macroblock.

Variants you'll see

Setting, x264 and x265How energy becomes an offset
aq-mode 1A straight line in the logarithm of energy, with a fixed zero point
aq-mode 2A curve whose reference point comes from the frame's own statistics
aq-mode 3Mode 2 plus a bias that lowers QP on low-energy blocks
aq-mode 4, x265 onlyMode 2, with edge density standing in for energy on blocks that contain edges

The rows describe broad behavior; the two encoders' constants differ, and so do their defaults (mode 1 in x264, mode 2 in x265). The exact x264 formulas are on Adaptive quantization in x264. The temporal counterpart is macroblock-tree (MB-tree), called coding-unit tree (CU-tree) in x265, which lowers QP on blocks that later frames reference. In both encoders its offset is applied on top of AQ's.

Trade-offs

Sources

Read 2026-11-13, at the commits linked.