Why filtering inside the loop helps prediction
A filter that cleans up decoded pictures could run after decoding, on the way to the display. The filters of this unit run earlier: on the reconstructed frame, before it is stored as a reference. Then the next frame is predicted from the filtered picture, and the encoder and decoder must apply exactly the same filter.
Two places to put a filter
After the loop, the filter is the player's business: the encoder never sees its output, so it cannot change what is coded. Inside the loop, the filtered frame is the one later frames are predicted from. Removing a reconstruction artifact from it can reduce the correction those predictions need; removing real detail can increase it, because the next residual is still the next source picture minus its prediction. Neither a saving nor lasting damage is guaranteed.
Compare the two on a moving cat
The cat walks a third of a pixel per frame for 25 frames. Frame 1 is predicted from flat mid-gray and every later frame from the reference of the frame before, with no motion search. Every frame is quantized at the step the quantization parameter (QP) nominally stands for. The filter is the deblocking filter of this unit. The table counts the nonzero levels the encoder produced over all 25 frames and averages the displayed picture's mean squared error against the source. A level is the integer quantization assigns to a coefficient; a count of nonzero levels is not a bitrate, and the bits each level takes are not measured. The Stored panel shows the selected frame as it is stored for predicting the next one, not the frame it was predicted from.
| Filter | Nonzero levels | Mean error |
|---|---|---|
| None | ||
| After the loop | ||
| Inside the loop |
For this cat at QP 34:
- Only the in-loop filter changed what was coded. It left 3,648 nonzero levels over the 25 frames, against 3,745 with no filter: 2.6% fewer. After the loop the count is 3,745 by construction. At QP 40 the in-loop count was 8.1% lower (1,536 against 1,672); at QP 22 the two were within 0.1% (10,056 and 10,063).
- Both placements lowered the average displayed error, the in-loop one by less. None 46.58, after the loop 43.65, inside the loop 45.24.
- Inside the loop, the filter's effect carried forward. On frame 25 the in-loop error was 47.66, above the unfiltered 41.35, and that frame used 141 levels against 128. Smoothing done on one frame is in the reference for the next, and this demo's encoder does not adapt its prediction or coding choices in response. This is not drift: both sides hold the same references.
So for this moving cat the in-loop filter lowered the count of nonzero levels and changed the displayed error: lower on average than with no filter, higher on the last frame. These are changes in a count, at unequal error, not bitrate savings. What a full encoder gains, with motion search and mode decision working on the filtered references, was not measured here.
What it commits both sides to
- The filter becomes part of decoding. When the filtered picture is used as a reference, both sides must rebuild the same reference pixels, as Why the encoder contains a decoder showed. A decoder that changed those pixels could make its later predictions differ from the encoder's. A filter after the loop can be anything the player likes.
- The encoder can account for it. In x264,
fdec_filter_rowdeblocks a finished row before the frame joins the reference list, and--psnrmeasures the filtered frame. With--no-deblock, x264 writes into each slice header that the filter is off, so the decoder does not run it either: the option changes the stream, not what a decoder may skip. - x265 and libaom also filter the reconstructed frame. x265 runs sample adaptive offset after deblocking on the reconstructed frame, and libaom runs its directional filter and loop restoration on the current frame's buffer.
Sources
Read 2026-11-15, at the commit linked. The filter code is cited on Deblocking and boundary strength.
- x264 at commit c24e06c (GitHub mirror):
fdec_filter_rowwith the deblocking call, the frame joining the reference list and the squared error for--psnr, the filter marked off for a slice and that mark written to the slice header;deblock_strength_c, whose rule the demo applies with equal references and motion (strength 2, strength 0); the clipping table, whose strength 2 column is in the demo. - Not checked: any specification text on in-loop filtering; whether x265 and libaom store the filtered frame as the reference (assumed from the order of their calls, not traced); the gain from in-loop filtering on real video with a full encoder, which this demo cannot measure; post-processing filters in players.