encoding.info
You are hereSourcePredictionTransformEntropy codingFiltersFramesDecisionsRate controlPerceptionQualityBitstreamSettingsEncoder map
Page 3 of 4 in In-loop filters

Directional enhancement and loop restoration in AV1

In-loop filters · implementation · added 2026-11-15

AV1 follows its deblocking filter with two more in-loop filters. The constrained directional enhancement filter (CDEF) reduces ringing with a directional filter: it weights neighbors along each block's direction most and limits the pull of neighbors with very different values, which helps preserve edges. Loop restoration then applies a filter whose parameters the encoder fits to the source. libaom runs them in that order: deblocking, CDEF, loop restoration.

Find the direction, then filter along it

For every 8×8 block, CDEF finds which of eight directions the block's pattern runs along. It works from the reconstructed pixels, so the decoder finds the same direction and nothing is sent. Each pixel is then nudged toward its neighbors: four primary taps along that direction, and eight weaker secondary taps along the two directions 45 degrees to either side.

Direction 0
Direction 1
Direction 2
Direction 3
Direction 4
Direction 5
Direction 6
Direction 7
The four primary taps for each direction, from libaom's table: the teal square is the pixel being filtered and the dark squares are the neighbors it reads. Direction 2 is horizontal and direction 6 vertical.

The constraint is what limits smoothing across a strong transition. A neighbor's pull is not its difference d from the pixel but a limited version of it:

shift = max(0, damping
            - log2(strength))
limit = max(0, strength
            - (|d| >> shift))
pull  = min(|d|, limit)

with the sign of d, and log2 rounded down; strength 0 gives no pull. At strength 8 and damping 6, a neighbor 4 levels away pulls by 4, one 16 away by 6, and one 64 away by nothing: small differences count in full, medium ones are limited, large ones are shut out. The filter measures only the difference; it does not know where an edge is.

Try the strengths on the cat

The cat is coded in 4×4 blocks with the Step slider's quantizer step, then filtered. Primary and Secondary are the two strengths, which the encoder chooses. For luma, libaom scales the primary strength in each block by how strongly directional the block is, and the demo does the same. Error is the mean squared error against the source: the average of the squared pixel differences. The Directions panel draws the direction found for each block over a faded copy of the reconstruction.

Primary at 0 turns off the primary taps, not the whole filter. With Secondary above 0, libaom then filters as for direction 0: the secondary taps lie horizontally and vertically, and the directions found go unused. The panel then draws them dashed and gray.

8
← 0: primary off15: strongest →
32
← finer quantizercoarser quantizer →
Source
Reconstructed
Filtered
Directions
Pixels changed
Error before
Error after
Simplified demo: the cat is invented, the transform is a floating-point 4×4 cosine transform, every block is predicted as flat mid-gray, and no deblocking filter runs first. The direction search, the taps, the constraint, the rounding and the per-block scaling of the primary strength are libaom's 8-bit luma code, with damping fixed at 6 (libaom sets it from 3 to 6 from the frame's quantizer). Not modeled: chroma, the blocks libaom skips, and how the encoder picks the strengths. Mean squared error does not measure how visible ringing is.

For this cat:

Loop restoration

The last filter works on larger restoration units, nominally up to 256 pixels across in libaom (a short remainder at the edge joins the last unit). Each plane of a frame has a restoration type: none, Wiener, self-guided or switchable. With one filter named, a unit uses it or nothing; with switchable, either filter or nothing.

CDEF finds its direction from reconstructed pixels, though its damping and strengths are sent. Restoration derives nothing: the encoder, whose search is handed the source frame, chooses every parameter and sends it, as with sample adaptive offset. No demo here models loop restoration.

The settings in libaom

Both descriptions are the option list's own text; the code behind them was not read.

Sources

Read 2026-11-15, at the commit linked.