Distributed Erlang compatibility is guaranteed for not just one, but two major versions. You can do a standard rolling upgrade and the newer nodes will talk to older nodes just fine.
It's great that they support C and JVM stack. But for other major platforms such as NodeJS, Go, ... we still need to approach the goal with some C extensions integrated in both platforms. Erlang/OTP distribution/clustering is really not designed for heterogeneous environment which is fine, since it is used in telecom industry with nice commercial support contracts backed by Ericsson. However I don't consider it as an amazing piece of engineering from a perspective of a system designer who is dealing with many teams with different language/stack preferences.
Erlang is what you would use to implement a single service (which may run on a cluster of multiple nodes). It's not a generic messaging layer to use between services, you can use RabbitMQ or ZeroMQ or whatever you like for that.
http://erlang.org/pipermail/erlang-questions/2011-June/05946...
Foreign nodes in C, Java, Python, etc. can also join an Erlang cluster:
http://erlang.org/doc/tutorial/cnode.html
http://erlang.org/doc/apps/jinterface/jinterface_users_guide...
That being said, a more typical architecture is to have Erlang spawn external processes and talk to them via stdio.