it seems to me that you're doing 2 factor on the ssh key decryption. That's not so useful IMO from the security point of view. The factor should be an alternative to the ssh key itself aka server side aka via ForceCommand as suggested