Directional enhancement and loop restoration in AV1
AV1 follows its deblocking filter with two more in-loop filters. The constrained directional enhancement filter (CDEF) reduces ringing with a directional filter: it weights neighbors along each block's direction most and limits the pull of neighbors with very different values, which helps preserve edges. Loop restoration then applies a filter whose parameters the encoder fits to the source. libaom runs them in that order: deblocking, CDEF, loop restoration.
Find the direction, then filter along it
For every 8×8 block, CDEF finds which of eight directions the block's pattern runs along. It works from the reconstructed pixels, so the decoder finds the same direction and nothing is sent. Each pixel is then nudged toward its neighbors: four primary taps along that direction, and eight weaker secondary taps along the two directions 45 degrees to either side.
The constraint is what limits smoothing across a strong transition. A neighbor's pull is not its difference d from the pixel but a limited version of it:
shift = max(0, damping
- log2(strength))
limit = max(0, strength
- (|d| >> shift))
pull = min(|d|, limit)
with the sign of d, and log2 rounded down; strength 0 gives no pull. At strength 8 and damping 6, a neighbor 4 levels away pulls by 4, one 16 away by 6, and one 64 away by nothing: small differences count in full, medium ones are limited, large ones are shut out. The filter measures only the difference; it does not know where an edge is.
Try the strengths on the cat
The cat is coded in 4×4 blocks with the Step slider's quantizer step, then filtered. Primary and Secondary are the two strengths, which the encoder chooses. For luma, libaom scales the primary strength in each block by how strongly directional the block is, and the demo does the same. Error is the mean squared error against the source: the average of the squared pixel differences. The Directions panel draws the direction found for each block over a faded copy of the reconstruction.
Primary at 0 turns off the primary taps, not the whole filter. With Secondary above 0, libaom then filters as for direction 0: the secondary taps lie horizontally and vertically, and the directions found go unused. The panel then draws them dashed and gray.
For this cat:
- At step 32, primary 8 and secondary 2 changed 556 of the 1,024 pixels, and the error fell from 42.38 to 34.10. With both strengths at 0 nothing changes. With Primary at 0 and Secondary at 2 the filter still changed 246 pixels (42.38 to 39.80).
- The directions came from the reconstruction and still matched the source. At step 32 all 16 blocks got the direction the source picture's blocks get.
- The strongest setting was not always the closest to the source. At step 32 the lowest error of the 64 settings was at primary 15, secondary 4 (32.04). At step 16 that setting gave 8.87, and primary 8 with secondary 4 gave 7.91.
Loop restoration
The last filter works on larger restoration units, nominally up to 256 pixels across in libaom (a short remainder at the edge joins the last unit). Each plane of a frame has a restoration type: none, Wiener, self-guided or switchable. With one filter named, a unit uses it or nothing; with switchable, either filter or nothing.
- Wiener. A separable filter: one set of weights applied horizontally and one vertically, seven taps for luma and five for chroma. The unit's weights are sent.
- Self-guided. The unit sends the index of one of 16 fixed parameter sets and up to two weights. The set gives the radius and strength for up to two filtered versions of the unit, each blending a pixel with its local mean, less where the neighborhood varies more; the weights mix them with the unfiltered pixel.
CDEF finds its direction from reconstructed pixels, though its damping and strengths are sent. Restoration derives nothing: the encoder, whose search is handed the source frame, chooses every parameter and sends it, as with sample adaptive offset. No demo here models loop restoration.
The settings in libaom
--enable-cdeftakes 0 to 3. The option list describes them as off, on (the default), off for frames that are not references, and chosen from the frame's quantizer.--enable-restorationtakes 0 or 1. The option list gives 1 as the default except in realtime mode.
Both descriptions are the option list's own text; the code behind them was not read.
Sources
Read 2026-11-15, at the commit linked.
- libaom at commit 1b7f9ef (GitHub mirror): in
av1/common/cdef_block.c, the direction table,cdef_find_dir_c, the primary and secondary tap weights, the primary taps and the secondary taps, the rounding,adjust_strength, where luma uses it, direction 0 passed when the primary strength is zero and the secondary taps two directions to either side;constraininav1/common/cdef.h; the damping, 3 plus the quantizer index divided by 64; inav1/encoder/encoder.c, the deblocking level chosen, then the CDEF search and filter, then the restoration search, given the source and filter; the restoration types, the seven-tap Wiener window, the largest unit and the 16 self-guided parameter sets, with the blend with the local mean, the two weights decoded and applied, the five-tap chroma window and a short last unit joined to the one before; inav1/decoder/decodeframe.c, the restoration type of each plane, the choice read for each unit, the Wiener weights and the self-guided set index and weights read, and the CDEF damping and strengths read;--enable-cdefand--enable-restorationin the option list. - Not checked: the AV1 specification text; AV1's deblocking filter; how libaom chooses the CDEF strengths, how a block selects among the strengths sent and which blocks it skips; the rest of the arithmetic of the Wiener and self-guided filters, how their parameters are coded and how the encoder searches for them; super-resolution, which libaom runs between CDEF and loop restoration when it is on; the code behind the two options; SVT-AV1; how visible the changes are.