TIL: JPEG XS - an image and video codec that offers both visually and mathematically lossless quality for low latency implementations.

Additionally, JPEG XS compressed content is indistinguishable from the original uncompressed content.

https://en.wikipedia.org/wiki/JPEG_XS

I've had great results using JPEG-XS to transport video for colour grading in feature film & TV post production. At 3:1 or 4:1 compression ratio is effectively lossless.

It is patent-encumbered though, you have to pay license fees to deploy it.

We use JXS when latency is critical. Most h24/265 decodes will have a 10 frame glass-glass delay, JXS drops that to 3 or 4, at a cost of bandwidth (our UHD jxs streams are 1.5gbit rather than 200mbit for hevc)

That's pretty depressing to read. x264 was handling the encoding side with sub-frame latency 15 years ago, and sub-frame decoding is significantly easier. "with –tune zerolatency, single-frame VBV, and intra refresh, x264 can achieve end-to-end latency (not including transport) of under 10 milliseconds for an 800×600 video stream"

But for some reason you can't make use of that and have to burn bandwidth instead.

A small part of the end-to-end process

https://www.obe.tv/how-to-lie-about-latency/

Bandwidth is cheap -- basically free, especially at this bitrate.

In theory it's a small part. But if you got that many frames of latency difference by changing codec, then it wasn't being a small part.

It's not that you should have gotten a magical 10ms latency glass to glass, it's that you should have been able to get 4 frames latency on h.264. But something prevented that, so I'm sad about it.

(And if you say the bandwidth was fine in your situation I won't argue, but using more than a gigabit extra is not usually thought of as free.)

Yeah, we've been deploying JPEG-XS for high bitrate streaming for a while.

A lot of our customers are moving their grading systems into data centres and streaming the images over IP back to their grading suites.

I've got it down to less than 1 frame for encode-transport-decode, but you've still got to copy the image to an SDI card and wait for that to clock out.

Isn't the point of JPEG to have lossy compression for your photos that still looks fine? As opposed to something like PNG, which has lossless compression

"JPEG" is short for Joint Photographic Experts Group, an ISO/ITU group that creates a lot of imaging standards. The JPEG image format you're thinking of is only one of the formats they've created.

The Joint Photographic Experts Group manages many standards, generally each called "JPEG [something]". The one we most commonly call "JPEG" is just one of them.

Reading that it looks like the point of JPEG-XS is to have near-lossless compression for raw photo and video data while having extremely high throughput.

JPEG XS supports either near lossless or truly lossless encoding depending on encoder configuration.

> Additionally, JPEG XS compressed content is indistinguishable from the original uncompressed content.

It can be indistinguishable, as long as you stick with lossless or very low compression ratios. It falls apart at typical JPEG XL compression ratios.

Not royalty free, unfortunately.

> gfxcapture: Windows.Graphics.Capture based window/monitor capture

> This source provides low overhead capture of application windows or entire monitors. The filter outputs hardware frames in d3d11 format; use hwdownload,format= if system memory frames are required.

This would strongly alter my plans if I were to develop an OSS Discord alternative. Chromium originally looked like a better core to start with largely due to its mature screen capture API. WebRTC is the other big thing, but there are other ways to do that. Native desktop apps (i.e., not browser based) are beginning to look much more compelling to me now.

If you were doing this, consider cribbing from https://github.com/obsproject/obs-studio/tree/master/plugins... which offers a variety of solutions including some rather exciting looking process injection (called "game" there).

I wonder if "entire chat app functions as OBS plugin" would work? Would solve the AV streaming side of the functionality.

You could always use Windows.Graphics.Capture directly.

Wait, are you https://www.gyan.dev/ffmpeg/builds/ ?

What I want to know is how much of these were written and/or debugged using AI tools and which ones? Using which workflow?

For that's an actual project, with countless uses, on countless machines.

Show me the AI. I want to see what AI has generated in those.

(btw I pay religiously my Claude Code subscription plan)