One way to do it is to whitelist all binaries in the system, and sandbox all applications (to prevent chances of a malicious PDF/image/etc abusing a buggy application).

can you do that on windows? every single exe, every process?

Yeah, the security policies let you do that. I think the current mechanism is called AppLocker.

Note that there may be still ways to bypass it if you're an attacker sitting at the computer, rather than a hapless user.