Depending on the encoder the GOP size / keyframe interval can be more or less unbounded as well. Typically you'd only have a few seconds per, but for example CCTV cameras are happy to produce minute-long GOPs. Seeking backwards in a video player tends to not work so well at that point, and most NLEs are significantly unhappy with such a file.
I wonder though if this could be solved with edit lists, supported e.g. by ISO BMFF. Not all players support them completely, though, for example I believe mpv only supports one edit, so I guess that's not useful for general purpose use..
