Publisher might want to display their URL in the address bar. But as a user I want to see the actual URL, not what Google or publisher want to show me. I don't want to see "example.com" in the address bar while I am actually connected to Google via a TLS connection signed by Google's key and my IP address is collected according to Google's privacy policy.
What confuses users is Google displaying a fake address bar [1] or browser displaying the wrong URL.
The URL you see _is_ the actual URL. It doesn't matter where the content was initially loaded from because the page is signed by the publisher's private key (the publisher has full control over the page contents, Google can't alter it).
The content is served from Google's servers according to Google's (not publisher's) privacy policy. While Google cannot alter the content, it sees the unencrypted HTTP request. I don't want neither Google nor publisher to control contents of my address bar.
Google already knows the unencrypted contents of the page, and they know you clicked on a link to it (from their search results page). The signed exchanges system doesn't reveal any information to Google _or_ the publisher that they don't already know.
Your browser controls the contents of the URL bar, not Google or the publisher.
What confuses users is Google displaying a fake address bar [1] or browser displaying the wrong URL.
[1] https://developers.google.com/search/docs/guides/images/amp0...