Like SPS/PPS they do contain information which will be needed to
decode the following data (as per definition of the flag)

Also ensures that the series of SPS/PPS/SEI NALU before a keyframe
can be considered as one contiguous header