Its worse than that. Cameras (plus DToF if you want tracking in the dark), imus, gyros, and necessarily onboard compute/SOC to handle processing that data. Shipping it all off to a remote computer and then making the round trip creates an untenable amount of lag. Thats not even accounting for controller and hand tracking.
And once you have the pipeline and computation power to enable inside out tracking all on device, adding an OS is essentially free.
It already has an IMU and gyro, obviously. Time of flight cameras are unnecessary. Steam Frame doesn't have them either. At most you would put IR LEDs for illumination which are tiny but also optional (Quest 3 doesn't have them), and there's no reason they have to be in the headset, you could just have a standalone IR illuminator on your desk.
As for sending data over a cable, there's nothing inherently laggy about it. After all, the display signal already travels over the cable, and the cable transfer is by far not the limiting factor in latency. The camera data is lower bandwidth than the display signal, too.
And once you have the pipeline and computation power to enable inside out tracking all on device, adding an OS is essentially free.