> in theory you could do e2ee accross multiple servers and devices

I use XMPP to send e2ee messages to friends on other servers and clients every day, so it's very much not just 'in theory'.