Don't send them upstream because for upstream a BYTES seek
might make sense but is completely wrong because upstream
can't seek to a byte position of the audio or video stream.

Also don't build the index in push mode for non-TIME seeks,
things will go wrong here otherwise.