encoding.info
You are hereSourcePredictionTransformEntropy codingFiltersFramesDecisionsRate controlPerceptionQualityBitstreamSettingsEncoder map
Page 4 of 4 in In-loop filters

Why filtering inside the loop helps prediction

In-loop filters · concept · added 2026-11-15

In Brief:
An in-loop filter runs on the reconstructed frame before it is stored as a reference, so later frames are predicted from the filtered picture. Removing artifacts there can reduce the correction those frames need, and the encoder and decoder must apply exactly the same filter.

A filter that cleans up decoded pictures could run after decoding, on the way to the display. The filters of this unit run earlier: on the reconstructed frame, before it is stored as a reference. Then the next frame is predicted from the filtered picture, and the encoder and decoder must apply exactly the same filter.

Two places to put a filter

After the loop: only the display is filtered
RebuildReferenceFilterDisplay
Inside the loop: the reference is filtered too
RebuildFilterReferenceDisplay
teal box: the filter gray boxes: the other steps
Conceptual illustration. "Rebuild" is the reconstruction of the hybrid codec loop: rescale, inverse transform, add the prediction. The arrow from the reference frames back to prediction is not drawn.

After the loop, the filter is the player's business: the encoder never sees its output, so it cannot change what is coded. Inside the loop, the filtered frame is the one later frames are predicted from. Removing a reconstruction artifact from it can reduce the correction those predictions need; removing real detail can increase it, because the next residual is still the next source picture minus its prediction. Neither a saving nor lasting damage is guaranteed.

Compare the two on a moving cat

The cat walks a third of a pixel per frame for 25 frames. Frame 1 is predicted from flat mid-gray and every later frame from the reference of the frame before, with no motion search. Every frame is quantized at the step the quantization parameter (QP) nominally stands for. The filter is the deblocking filter of this unit. The table counts the nonzero levels the encoder produced over all 25 frames and averages the displayed picture's mean squared error against the source. A level is the integer quantization assigns to a coefficient; a count of nonzero levels is not a bitrate, and the bits each level takes are not measured. The Stored panel shows the selected frame as it is stored for predicting the next one, not the frame it was predicted from.

34
← finer quantizercoarser quantizer →
25
← first framelast frame →
Source
Displayed
Stored
FilterNonzero levelsMean error
None
After the loop
Inside the loop
Frame
Levels, this frame
Error, this frame
Simplified demo: the cat and its motion are invented, the transform is a floating-point 4×4 cosine transform, and QP sets the quantizer step it nominally stands for and the filter's thresholds. The filter arithmetic and tables are x264's for luma, with offsets 0:0. Frame 1 is filtered as intra. In later frames an edge is filtered at strength 2 when either block has nonzero levels and is skipped otherwise, which is what x264's strength rule gives when both blocks use the same reference and motion. The table's three rows are for the QP shown; the tiles and pictures follow the Filter and Frame controls. Later residuals do respond to the filtered reference, which is why the counts differ. But the predictor rule, the QP and the filter settings stay fixed: there is no motion search, no mode decision and no rate control looking for a better result.

For this cat at QP 34:

So for this moving cat the in-loop filter lowered the count of nonzero levels and changed the displayed error: lower on average than with no filter, higher on the last frame. These are changes in a count, at unequal error, not bitrate savings. What a full encoder gains, with motion search and mode decision working on the filtered references, was not measured here.

What it commits both sides to

Sources

Read 2026-11-15, at the commit linked. The filter code is cited on Deblocking and boundary strength.