I think there is a token being passed, but it's authentication, not identity. It's a signed declaration that you own that identity, and it must be either from the domain that issued it, or another source that the site you're visiting specifically trusts (i.e. Mozilla, at present).
I trust Mozilla to have done a good job of this. If there's a weakness, it's not going to be anything so obvious that you can see it from my stumbling attempts to describe the protocol. If it sounds like there is, I'm probably describing it wrong. Have a look at the details here: http://lloyd.io/how-browserid-works
There is a token that's passed. The web site gets an email address and a string called "an assertion" that they must verify.
If Gmail suddenly started verifying instead of Persona for @gmail.com addresses, the web site would see the email address as exactly the same so should give access to the same account.
They would then start verifying that "assertion" using Gmail and not Persona. It would be verified and hence secure.
I haven't thought it all through, but it feels like there's a weakness there that's just waiting to be exploited.