Eh, I use a large number of the top plugins in the OP, but I've never had any trouble dropping back to "vanilla" vi/m in ssh shells or whatever. It's just a matter of convenience; there's nothing there that's so totally transformational that you won't be able to use vim without it.