It's not hard though, it's just hard to do it without any third-party in a way that provides an easy user experience. iMessage is a good example of encrypted E2E messaging that "just works", but it is setup and maintained by a third party.
I would say one of the biggest hurdles to increased security is the debilitating nature of secure from the security people themselves. Everyone wants a perfect solution and postulates endlessly about every edge case. So much so that the community won't accept any solution that has even a little bit of hair on it, so nothing gets done except everyones individual homebrew mechanisms. I swear there's not a pragmatic bone in their bodies, and if you doubt me then join some popular crypto mailing lists and see the tinfoil hattery for yourselves.
I would say one of the biggest hurdles to increased security is the debilitating nature of secure from the security people themselves. Everyone wants a perfect solution and postulates endlessly about every edge case. So much so that the community won't accept any solution that has even a little bit of hair on it, so nothing gets done except everyones individual homebrew mechanisms. I swear there's not a pragmatic bone in their bodies, and if you doubt me then join some popular crypto mailing lists and see the tinfoil hattery for yourselves.