encoding.info
You are hereSourcePredictionTransformEntropy codingFiltersFramesDecisionsRate controlPerceptionQualityBitstreamSettingsEncoder map
Page 3 of 4 in Frame structure

Hierarchical B-frames

Frame structure · concept · added 2026-11-15

In Brief:
A hierarchy of B frames lets some B frames serve as references for others, which shortens many prediction distances. Those reference frames must be decoded first, which changes the decoding order and how long frames have to stay in memory.

An ordinary B frame is predicted from the I or P frames around it and is used by nothing. With several B frames in a row, the ones in the middle are far from both of their references. A hierarchy fixes that by letting some B frames serve as references for the others: code the middle frame first, then predict the frames on each side from it.

Compare the structures

Nine frames of the cat's walk are coded at quantization parameter (QP) 28. Frame 1 is intra, and a P frame closes each run of B frames, predicted from the I or P frame before it. Choose how many B frames sit in a run and which of them are references. In the drawing, left to right is display order. I and P frames are on the top row, and each row below holds B frames predicted from frames on rows above; each B frame hangs from a line that runs between its two references. The Coded order readout gives the order they are coded in. A filled teal box is a B frame that other frames use. The table counts nonzero levels over all nine frames and averages the mean squared error against the source. Its first row is the same nine frames with no B frames at all.

coral: I frame gray: P frames filled teal: reference B frames outlined teal: other B frames dark outline: the chosen frame
StructureNonzero levelsMean error
No B frames
No reference B
The middle one
Every level
5
← first shownlast shown →
Source
Prediction
Reconstructed
Coded order
Levels, this frame
Predicted from
Simplified demo: the cat and its motion are invented; the transform is a floating-point 4×4 cosine transform at the nominal H.264 step for QP 28, the same for every frame. There is no motion search. A B frame block takes one of its two references or their average weighted by distance in time, whichever is closest to the source. "The middle one" follows x264's rule for where the reference B frame goes; "Every level" keeps halving each gap. With 3 B frames those two are the same structure, and with 1 there is nothing to choose. A count of nonzero levels is not a bitrate.

For this cat at QP 28:

What x264 does

In its normal automatic placement, x264's --b-pyramid is one level deep. When a run has more than one B frame and none of them is a reference already, x264 makes the middle one a reference and leaves the rest ordinary. The option takes none, strict or normal, and normal is the default; strict marks the reference B frame at a lower priority than I and P frames. With the pyramid on, x264 sets its B frame reordering delay to two frames where plain B frames set one; its total input delay is a separate figure that also counts the lookahead and threads. x265 has --b-pyramid on by default too; its structure was not read. The demo's "Every level" is not x264's automatic structure, and no encoder with a deeper hierarchy was read for this page.

Trade-offs

Sources

Read 2026-11-15, at the commits linked.