Multiple reference frames
The previous frame is the obvious place to look for a prediction, and it is not always the best one. Something may have been hidden a frame ago and visible before that. With multiple reference frames the encoder can choose, block by block, among several reconstructed pictures that the decoder also holds, and the stream identifies the one used, explicitly or by inference. This page uses references from earlier in display order; bidirectional prediction is a later page.
Try giving the encoder a longer memory
Choose a scene and how many earlier frames the encoder may use. For every 8×8 block of the current frame, the demo searches each allowed frame and keeps the match with the lowest score. The score is the sum of absolute differences (SAD), which adds up the absolute difference between each sample and its prediction, plus an invented penalty of 24 for each frame further back. So the demo can prefer a slightly worse match from a newer frame. The readouts show the SAD of the chosen predictions, without the penalty. The residual panel is mid-gray where the prediction is exact.
- The bus scene gains from the frame from before the bus arrived. SAD is 14,709 with one frame, 3,952 with two and 1,964 with three; a fourth changes no block's choice. The poster is mostly covered one frame back, half covered two back and clear three back.
- The blinking sign gets its large gain from two. The sign's last lit frame is two back: SAD falls from 18,113 to 2,033, and no block changes after that.
- On the pan, the penalty keeps every block on the nearest frame. Ranked by SAD alone, older frames would lower it a little, from 4,891 to 4,658 with four frames, but in no block by enough to outweigh the penalty, so SAD stays at 4,891. More candidates need not change the result once naming them has a cost.
Reference frames in x264 and x265
--ref sets the most reference frames a frame may choose from. Both option parsers accept 1 to 16, but the usable count can be lower: x265 needs the count plus the number of reordered pictures to be at most 8 to fit any level, and with a level requested it lowers --ref to fit. The preset values, before tunings and other options:
| Preset | x264 | x265 |
|---|---|---|
| ultrafast, superfast | 1 | 1 |
| veryfast | 1 | 2 |
| faster | 2 | 2 |
| fast | 2 | 3 |
| medium | 3 | 3 |
| slow | 5 | 4 |
| slower | 8 | 5 |
| veryslow, placebo | 16 | 5 |
x264's fast search adds an estimated cost for the reference index to each candidate: lambda times a bit count from a table. With one active reference it is zero; with two, 1 bit for either; with three or more, 1 bit for the first entry of the reference list, 3 for the next two and 5 for the four after. x264 orders the list by distance, so the first entry is normally the nearest frame. This is an estimate, not a fixed bill: what is written depends on the mode and the entropy coder, and some modes write no index at all.
x264's --mixed-refs, on in the medium preset, lets each 8×8 quarter of a macroblock use a different frame. Without it the quarters reuse the frame chosen for the whole macroblock. It is off in the four fastest presets and has no effect when --ref is 1.
Trade-offs
- More allowed references can mean more search. A motion search can be repeated for each reference, so encoding time tends to rise with the count. Encoders prune, and by how much is not measured here.
- The decoder must keep every picture the stream may point at. That is memory in each playback device, and it is why standards limit the count by level. The limits were not checked for this page.
- The gain depends on the content. In the demo it is large when something is uncovered or repeats, and nothing for the pan under this score. The numbers are for these three scenes only.
- Naming the reference can cost bits. Where an index is coded it adds to the block's cost, even for blocks that use the nearest frame; where the choice is inferred it does not.
Sources
Read 2026-11-13, at the commits linked.
- x264 at commit c24e06c (GitHub mirror): the default, the presets, the 1 to 16 limit, the reference cost table and which row of it is used,
bs_size_te, the index length, the 8×8 search with mixed references and without, mixed references off with one frame, the list ordered by distance, the macroblock type that writes no indexes, the arithmetic coder's context-dependent index. - x265 at commit 4191822: the default, the presets, the 1 to 16 limit, the limit of 8 when detecting a level, lowering
--reffor a requested level and for its picture buffer. - Not checked: the level limits on reference count in either standard, whether x264 lowers
--reffor a level, how reference lists are reordered, how tunings change--ref, x265's--limit-refs, bidirectional frames, weighted prediction and long-term references. The trade-offs are general reasoning, not measurements.