It's completely normal for some decoders to queue 50-60 frames without it causing any problems, e.g. RPi.