encoding.info
You are hereSourcePredictionTransformEntropy codingFiltersFramesDecisionsRate controlPerceptionQualityBitstreamSettingsEncoder map
Page 5 of 5 in Prediction

Multiple reference frames

Prediction · concept · added 2026-11-13

In Brief:
The previous frame is not always the best one to predict from. Letting each block choose among several stored pictures helps when something was hidden a frame ago or repeats, at a cost in search time and decoder memory.

The previous frame is the obvious place to look for a prediction, and it is not always the best one. Something may have been hidden a frame ago and visible before that. With multiple reference frames the encoder can choose, block by block, among several reconstructed pictures that the decoder also holds, and the stream identifies the one used, explicitly or by inference. This page uses references from earlier in display order; bidirectional prediction is a later page.

Try giving the encoder a longer memory

Choose a scene and how many earlier frames the encoder may use. For every 8×8 block of the current frame, the demo searches each allowed frame and keeps the match with the lowest score. The score is the sum of absolute differences (SAD), which adds up the absolute difference between each sample and its prediction, plus an invented penalty of 24 for each frame further back. So the demo can prefer a slightly worse match from a newer frame. The readouts show the SAD of the chosen predictions, without the penalty. The residual panel is mid-gray where the prediction is exact.

1
← only the previous framethe last four →
1 back
2 back
3 back
4 back
Current
Prediction
Residual, ×2
Frame used
1 back 2 back 3 back 4 back. Frames the encoder may not use are faded.
Frames allowed
SAD, whole frame
Blocks using an older frame
Simplified demo: the scenes are invented source pictures, 64×32 pixels, with a little independent noise in every frame; an encoder searches reconstructed references. Each of the 32 blocks is searched sideways only, up to 48 whole pixels, in each allowed frame; past the frame edge the edge pixel repeats. The score is SAD plus 24 per frame further back, plus 0.01 per pixel of vector length, which is too small to do more than settle ties toward the shorter vector. The penalty stands in for the cost of naming an older frame; real encoders estimate that cost in bits, and the vector's as well. Residual grays are amplified two times and clipped to the display range; SAD uses the full differences.

Reference frames in x264 and x265

--ref sets the most reference frames a frame may choose from. Both option parsers accept 1 to 16, but the usable count can be lower: x265 needs the count plus the number of reordered pictures to be at most 8 to fit any level, and with a level requested it lowers --ref to fit. The preset values, before tunings and other options:

Presetx264x265
ultrafast, superfast11
veryfast12
faster22
fast23
medium33
slow54
slower85
veryslow, placebo165

x264's fast search adds an estimated cost for the reference index to each candidate: lambda times a bit count from a table. With one active reference it is zero; with two, 1 bit for either; with three or more, 1 bit for the first entry of the reference list, 3 for the next two and 5 for the four after. x264 orders the list by distance, so the first entry is normally the nearest frame. This is an estimate, not a fixed bill: what is written depends on the mode and the entropy coder, and some modes write no index at all.

x264's --mixed-refs, on in the medium preset, lets each 8×8 quarter of a macroblock use a different frame. Without it the quarters reuse the frame chosen for the whole macroblock. It is off in the four fastest presets and has no effect when --ref is 1.

Trade-offs

Sources

Read 2026-11-13, at the commits linked.