encoding.info
You are hereSourcePredictionTransformEntropy codingFiltersFramesDecisionsRate controlPerceptionQualityBitstreamSettingsEncoder map
Page 3 of 3 in Codec by codec

Bitstream structure

Codec by codec · concept · added 2026-11-15

In Brief:
Encoded video is packaged into units, each with a small header and a marked boundary. That lets a reader locate and classify data, but a unit may still depend on settings and earlier frames. Starting or recovering playback also needs the right decoding state.

An encoder's output is not one long string of coded blocks. It is a sequence of separately identifiable units, each with a small header that says what it holds: the settings a decoder needs first, a coded picture or part of one, or extra information. The unit layer helps a player find a place to start, skip what it cannot use and pick up after an error; decoding from a unit still depends on the settings and reference frames the decoder holds.

Wrap a payload as a unit

The payload is twelve invented bytes standing in for coded data. Choose a format and what the unit carries, and the demo wraps the payload the way that format's encoder does. H.264 and High Efficiency Video Coding (HEVC) call the unit a network abstraction layer (NAL) unit and, in a plain stream, mark where each begins with a start code: two or three zero bytes and a one. The payload must then never contain that pattern, so the encoder inserts an escape byte, 03, wherever two zeros are followed by a byte of 3 or less. AV1 calls the unit an open bitstream unit (OBU); in the form libaom writes, a size field takes the place of a start code. One choice of unit is a slice of an instantaneous decoding refresh (IDR) picture, a kind of keyframe.

Gray: start codeTeal: header and sizeCoral: inserted escape bytes

Bytes written
Added to the payload
Escape bytes
Simplified demo: the payloads are invented and are not valid coded data. The start code lengths, header bytes and escape rule are those of x264 and x265, and the header and size bytes are those of libaom. For H.264 the last two choices of unit are x264's P-frame and its unreferenced B-frame; for HEVC they are the types x265 names TRAIL_R and TRAIL_N. libaom writes a whole frame as one unit type here, so the last three choices give the same header. These are the wrappers for the cases shown, not every branch of the writers: x264's three-byte start code assumes the IDR slice follows its parameter sets in the same output, and its four-byte one that the picture is the first unit of its frame's output, which a delimiter in front would change; x265 also appends 03 to a payload that ends in a zero byte, which none of these does.

For x264's sequence settings and the payload that contains 00 00 01 twice:

What the units are

The settings travel in parameter sets: a sequence parameter set (SPS), a picture parameter set (PPS) and, in HEVC, a video parameter set (VPS) above them. The numbers in the table are the type values in x264, x265 and libaom.

HoldsH.264HEVCAV1
Settings for the sequenceSPS, type 7VPS 32, SPS 33Sequence header, 1
Settings for picturesPPS, type 8PPS, type 34In the frame header
A coded picture or partSlice, 1 or 5Slice, 0 to 9, 16 to 21Frame, type 6
Start of a time instantDelimiter, 9Delimiter, 35Delimiter, 2

These are types of unit, not the I, P and B types of a slice's prediction.

The slice and frame types in detail

An H.264 NAL unit of type 5 carries a slice of an IDR picture: a keyframe at which x264 clears its list of references. HEVC defines more NAL unit types for coded slices; x265 lists values 0 to 9 and 16 to 21, and what each one means was not read for this page. For AV1 the table shows one way of packaging: libaom's list also has a frame header unit, 3, and a tile group unit, 4.

In the encoders

x264 repeats its parameter sets at every keyframe unless told not to, x265 only when asked, and libaom writes a sequence header at its key frames. A start code can give way to a length: x264 writes lengths when its output is a container.

Repeated settings, delimiters and container output, by encoder
  • Where the settings are repeated. x264's default puts them at hand wherever a stream is joined at a keyframe; with open groups of pictures that keyframe can be an I frame with a recovery point, not an IDR picture. x265's --repeat-headers is off by default. libaom's sequence header also goes before an intra-only frame.
  • Delimiters. They are optional in x264 and x265 (--aud, off by default). libaom puts a temporal delimiter in front of the frame data it returns, for spatial layer 0 and when no earlier data is waiting to go out.
  • Container output. For MP4, Matroska or Flash video x264 turns start codes and repeated headers off and puts a four-byte length before each unit; the escape bytes stay.

Trade-offs

Sources

Read 2026-11-15, at the commits linked. The lines on the non-IDR keyframe, x265's final byte, libaom's delimiter condition and its frame header were read 2026-11-16.