IPv6 is perfectly backwards compatible with IPv4 with the IPv4-in-IPv6-address embedding and other technologies.
The problem is that IPv4 is not forward-compatible with anything that has a larger address space. Thus a IPv4-only host will never ever be able to communicate with a non-IPv4 host, since there is no way to encode more than 32-bit of information in the IPv4 header.
So you will always end up in this situation where people just won't bother implementing the IPv4-replacement and you cannot simply switch to it.
But why couldn't we just put a middlebox in front of the host which translates a 4-over-6 address to a plain ipv4? The host box still sees only v4 but v6-only devices can still connect to it.
Like I could envision a simple router-like device doing this.
That's not the direction that's the problem - like I mentioned (IPv6 is backwards compatible), NAT64 and so forth boxes exist (but still not great since you have to hold state).
But the problem is that the inverse is not possible. How would a IPv4-only box connect to a non-IPv4 box? How do you encode more than 32-bit of information into the 32-bit destination address field of the IPv4 packet?
So you will always end up in this situation where people just won't bother implementing the IPv4-replacement and you cannot simply switch to it.