> It's still annoying to need to change the entire input method

But you are effectively changing the input method from "typing" to "gaming", so it makes sense to have to toggle some switch somewhere.