There's off-the-shelf semiconductors that perform the conversion in hardware, so all the dongles need is a simple breakout board for the component.

Crazy times

Usually slow

In what regard? The common Macrosilicon chip MS2130 has <100ms.

10 fps isn’t great, latency wise. Even typing or using a mouse is very clunky.

I may have been unlucky, but I have seen even worse.

Latency, not FPS. Still suitable for using it as KVM.

100ms latency, they do at least 30fps.